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INTRODUCTION 


STATEMENT  OF  RESEARCH  PROBLEM 

To  date,  control  design  is  generally  based  on  an  analysis 
approach.  That  is  to  say,  the  system  is  first  constructed  and  then 
analyzed  to  see  if  the  performance  is  satisfactory.  In  cases  where 
unsatisfactory  performance  is  incurred,  the  system  is  modified  in 
some  manner  and  then  re-analyzed.  This  procedure  may  be  satisfactory 
in  cases  where  there  are  few  control  parameters  to  manipulate.  Cases 
occur,  however,  where  several  independent  forcing  functions  may 
exist.  It  may  also  be  the  case,  that  a  nonlinear  controller  could 
out  perform  a  linear  controller. 

This  research  is  concerned  with  the  problem  of  formulating 
and  computing  optimal  control  laws  for  the  control  of  more  complex 
dynamic  systems.  The  optimization  of  a  dynamic  system,  subject  to 
a  given  performance  criterion,  and  a  known  mathematical  description 
of  the  plant  and  its  surroundings,  is  carried  out. 

There  is  available,  a  number  of  techniques  for  evaluating 
optimal  control  laws.  This  information  is  generally  found  in 
mathematical  and  engineering  papers  published  in  various  journals. 

To  date,  concise,  complete  results  are  difficult  to  find  for  the 
continuous- time  case.  The  purpose  of  this  paper  is  to  consolidate, 
extend  and  compare  results  of  modern  ideas,  in  the  control  field 
of  a  dynamic  system  under  a  unified  approach.  A  complete  analysis 
is  carried  out  using  classical  methods  of  calculus  of  variations. 
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2. 


STATEMENT  OF  THE  GENERALIZED  CONTROL  PROBLEM 

A  dynamic  system  is  one  whose  future  behaviour  depends  on 
its  past,  as  well  as  on  its  present  and  future  inputs.  The  influence 
of  a  system's  past,  can  be  specified  by  a  set  of  parameters  which  are 
termed  the  state  variables  to  determine  the  future  behaviour  of  the 
system.  The  outputs  can  be  expressed  as  a  function  of  the  state 
variables  so  that  if  x(s)  represent  the  system  states,  u(s),  the 
control  inputs,  y(s)  the  outputs  and  w(s)  the  disturbances,  then 
the  dynamic  system  can  be  described  by  the  following  set  of 
differential  equations 

dx(s)  =  f  (  s;x(s) ,u(s) ,w(s)) 
ds 

y(s)  -  h(£s)  ,x(s)w(s)) 

The  control  of  a  dynamic  system  is  a  matter  of  varying 
u(s)  in  such  a  manner  that  the  behaviour  of  the  system  is  in  some 
sense  best.  This  optimization  is  carried  out  to  optimize  a  specified 
performance  criterion. 

Performance  criterions  in  the  past  have  been  used  to  specify 
error  constants  in  a  steady  state  situation.  We  now  generalize  the 
performance  criterion,  to  be  given  by  the  following  integral  function 
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where  t  is  the  initial  time  of  the  system  and  T  is  the  terminal 
time. .  In  cases  where  stochastic  inputs  and  disturbances  are 
considered  the  expected  value  of  V( s ; x(s) u(s))  must  be  specified. 
L(s ;T, u(s) ,y(s) ,w(s))  is  called  the  loss  function  or  cost  function 
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of  the  dynamic  system.  It  is  not  necessary  that  T  be  fixed;  it  may 
rather  be  that  time  at  which  a  condition  of  the  form 

g(T,  x(s))  =  0  becomes  satisfied. 

The  inputs  to  the  controller  can  be  considered  a  set  of 
parameters  u(t)  which  are  statistically  related  to  w(s),  t<s<T 
and  make  central  decisions  on  that  basis. 

Since  the  system  behaviour,  as  characterized  by  x(s), 
t<s<T,  depends  on  x(t)  as  well  as  u(s)  and  w(s),  t<s<T,  the 
control  law  k  specifying  u(t)  must  be  a  function  of  x(t)  as  well 
as  u(t),  t  and  possibly  T: 

u(t)  -  k(t,x(t)  ,  V(t)  ,T) 

The  problem  is  to  select  the  control  law  which  maintains 
along  the  entire  trajectory  x(s),  t<s<T  the  maximum  performance 
criterion. 

above 

The  specialized  control  problem  described  alone  is 
represented  in  Figure  1.  It  represents  the  control  of  a  nonlinear, 
time-varying  multi-input,  multi-output  dynamic  system  subject  to 
stocl astic  disturbance. 

CONTROL  AND  OPTIMIZATION  OF  DYNAMIC  SYSTEMS  -  THE  VARIATIONAL 

APPROACH 

In  this  report,  the  variational  approach  is  used  to  determine 
optimal  controllers  for  a  dynamic  system.  In  Section  I,  Part  I  basic 
necessary  conditions  are  developed  for  the  generalized  control  problem 
using  the  variational  approach  to  optimize  the  performance  criterion 
subject  to  the  constraints  of  the  plant.  These  basic  necessary 
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conditions  are  just  the  Euler  Lagrange  equations  of  the  classical 
variational  problem  in  n  dimensions.  In  addition,  boundary 
conditions  which  the  optimal  system  must  obey  are  developed.  A 
second  necessary  condition  is  developed  following  the  classical 
ideas  of  Weierstrass  and  forming  the  well-known  Weierstrass  E  - 
function.  This  in  turn,  by  using  the  quadratic  remainder  of  the 
Taylor  series,  gives  Legendres  well-known  necessary  condition. 
Using  a  Legendre  transformation,  the  Euler  Lagrange  equations  are 
reduced  to  a  canonical  form  which  in  turn  must  obey  the  Hamilton 
Jacobi  partial  differential  equation  given  in  Section  I,  Part  2. 
Hence  an  additional  necessary  condition  for  optimum  is  that  the 
variation  of  the  Hamiltonian  be  equal  to  zero. 

The  concepts  of  controllability  and  observability  are 
developed  in  Section  I,  Part  2.  Controllability  and  observability 
are  sufficiency  conditions  for  a  linear  time  varying  system.  These 
sufficiency  conditions  can  be  thought  of  in  the  same  light  as 
stability  of  a  dynamic  system. 

The  Euler  Lagrange  equations  form  a  system  in  general  of 
nonlinear  differential  equations.  Because  of  this,  and  because  of 
the  fact  that  virtually  no  theory  has  been  developed  for  the 
general  solutions  of  nonlinear  equations,  we  are  unable  to  solve 
the  control  problem  in  a  general  sense.  If  nonlinear  equations 
were  encountered,  digital  techniques  would  have  to  be  employed  and 
each  problem  would  have  to  be  assessed  and  solved  on  its  own  merit 
Although  we  cannot  solve  nonlinear  systems  in  general,  solutions 


are  available  for  linear  time  varying  systems.  These  solutions  are 
given  in  Section  II,  Part  1  and  2,  where  the  solution  of  Part  1 
gives  a  new  method  of  solving  stationary  linear  problems  and  Part  2 
gives  a  method  for  solving  non-stationary  linear  problems.  The 
computational  advantage,  and  the  more  general  solution  (i.e., 
transients  considered^of  these  methods  over  the  older  methods  of 
using  transform  techniques,  provide  an  excellent  method  of 
solving  complex  problems.  These  methods  are  particularly  suited 
to  digital  techniques,  although  not  treated  as  such  in  this  report. 

The  solution  of  a  linear  system  in  no  way  guarantees  that 

the  control  law  is  linear.  Cases  where  linear  systems  are  governed 

by  nonlinear  controllers  are  illustrated  in  Section  II,  Part  4. 

It  is  possible,  however,  with  an  additional  assumption  to  the 

system  (i.e.,  a  quadratic  loss  function  of  the  form  given  in 

to 

Section  II,  Part  1  and  2)  form  a  controller  which  must  be  linear. 
Having  formed  the  optimal  control  problem,  as  done  in  Section  II, 
Part  1  and  2,  there  are  several  possible  ways  to  solve  this  problem. 
If  the  initial  time  t  is  held  fixed  or  specified  for  some  value  and 
successive  values  of  T  are  investigated,  the  control  law  can  be 
observed  as  it  approaches  its  limit.  This  is  known  as  the  Floating 
time  to  go  problem.  If  we  now  consider  a  fixed  value  of  T,  and 
successive  values  of  t,  another  solution  known  as  the  Shrinking 
time  to  go  problem,  can  be  solved. 
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In  Section  II,  Part  3,  a  somewhat  more  elegant  method  of 

solving  the  Shrinking  time  to  go  problem  subject  to  natural 

boundary  conditions,  is  suggested  by  using  a  method  formulated 

by  Kalman.  This  method  assumes  that  a  linear  relationship  between 

%  (t)  and  x(t)  exists  and  holds  true  at  t  =  T.  If  this  is  the 

case,  a  differential  equation  can  be  formed  for  P(t)  where  J!(t)  = 

P(t)x(t).  This  differential  equation  assumes  the  well-known  form 

of  the  Riccati  equation  and  shall  be  denoted  as  such  throughout  this 

paper.  Kalman's  technique  when  applicable,  has  the  advantage  that 

we  can  treat  the  two-point  boundary  problem  as  an  initial  value 

problem,  hence,  well-known  theory  of  ordinary  differential 

equations  can  be  applied.  Also  developed  for  this  problem  is  a 

very  useful  stability  theorem.  This  theorem  states  that  if  in 

addition  to  the  conditions  of  the  problem  as  stated  in  Section  II, 

Part  3,  the  plant  is  also  uniformly  completely  controllable  and 

uniformly  completely  observable,  then  the  optimal  regulator  is 

asymptotically  stable.  Thus,  rather  than  check  the  stability  of  the 

regulator,  it  is  only  necessary  to  check  whether  or  not  the  plant  is 

uniformly,  completely  controllable  and  observable.  In  addition, 

Kalman's  technique  provides  a  simple  way  in  which  to  calculate 

solutions  for  steady  state  problems.  This  is  done  by  equating  the 

to  zero 

right-hand  side  of  the  Riccati  equationAand  solving  the  resulting 
quadratic,  algebraic  equations  for  the  nonnegative  definite  solution. 
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The  solution  of  these  algebraic  equations  is  particularly  suited  to 
analog  techniques.  Further  as  shown  in  Section  III,  Part  1  and  2, 
the  regulator  problem  is  under  duality  relations  equivalent  to  the 
linear  filter  and  prediction  problem. 

In  Section  II,  Part  4,  methods  of  treating  constraints  for 
a  dynamic  system  are  considered.  In  addition  necessary  conditions 
for  a  linear  plant  subject  to  the  constraint  |  uj|<  1  are 
developed  for  minimum  time,  fuel  and  energy.  The  minimum  fuel 
problem  is  then  solved  completely  by  use  of  a  phase  plane  for  a 
second  order  system. 

In  Section  III,  the  linear  filter  problem  is  treated  from 
a  state  transition  method.  The  well-known  Wiener-Hopf  equations 
are  reduced  to  a  system  of  differential  equations.  In  addition  to 
being  able  to  solve  time  varying  systems,  this  method  provides  an 
easy  computational  way  to  solve  the  more  well-known  stationary 
problem.  In  Example  Problem  III  1-1,  a  stationary  filter  is  solved. 
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9. 


SECTION  I 

Part  I  A  BASIC  NECESSARY  CONDITION 

The  most  general  control  problem  is  investigated  and  a  basic 
necessary  condition  is  developed  for  an  optimal  plant.  The  plant 
is  described  by  the  non-linear  vector  differential  equation  (1.1). 

dxlsl  =  f (s;x(s) ,u(s) ,w(s))  . . . (1.1) 

ds 

where  f (s;x(s) ,u(s) ,w(s))  is  an  n-dimensional  vector  function. 
x(s)  is  an  n-dimensional  vector. 

u(s)  is  an  m -dimensional  vector  where  1  <  m  <  n 
w(s)  is  an  r -dimensional  vector  which  defines  the  disturbances 
that  occur  in  the  plant. 

The  output  of  the  plant  y(s)  is  given  by  (1.2) 

y (s)  =  h(s ;x(s)  ,w(s) )  . (1.2) 

where  y(s)  is  an  p -dimensional  vector. 

The  general  performance  criterion  "V"  is  specified  for  some 
future  time  from  the  present  time  "t"  to  that  future  time  "Tn 
(possibly  infinity),  is  of  the  form. 

V  -  ^  J  L(s;T,x(s) ,u(s) ,w(s))ds  +  £/(T,x(T))  . (1.2) 

where  "L"  and  ")J"  are  scalar  functions  of  the  vectors  x(s)  ,  u(s)  ,and  w(s). 

It  should  be  noted  at  this  point  that  the  vector  function 
h(s ;x(s) ,w(s) )  is  incorporated  into  the  performance  criterion. 

In  general  we  may  specify  a  boundary  condition  at  time  "T". 
g(T , x(T) )  -0  . (1.4) 
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The  current  system  state  x(t)  is  assumed  known  and  the  controller 
input  v(t)  is  considered  to  specify  completely  w(t)  for  all  t  <  s  <  T  . 
For  a  given  x(t)  and  v(t)  it  is  required  to  find  the  function  u(t)  for 
all  t  <  s  <  T  which  maximizes  "V".  The  control  law  "k"  specifying  u(s) 
must  be  a  function  of  x(s)  as  well  as  v(s),  s  and  possibly  T. 

Therefore  let 


(1.5) 


u ( s)  -  k(s ;x(s) ,v(s) ,T) 


where  "k”  is  a  vector  function  and  is  the  optimal  control  law  sought. 
Conditions  on  the  system 

(1)  u(s)  is  a  function  of  class  D°. 

(2)  f (s;x(s) ,u(s) ,w(s))  is  of  class  C'  with  respect  to  x(s)  and 
u(s)  is  of  class  D°  with  respect  to  "sM  in  the  region  R  of  interest  in 
the  s,x(s),u(s)  space. 

(3)  L  is  of  class  C'  with  respect  to  x(s)  and  u(s)  and  of  class 
D°  with  respect  to  "s"  in  R. 

Note:  L  will  later  become  of  class  with  respect  to  x(s)  and 
u(s)  due  to  Legendre’s  necessary  condition. 

(4)  The  system  is  to  be  controllable  with  respect  to  the  inputs 
u(s)  in  R. 

Note:  The  term  controllable  and  conditions  necessary  for 
controllable  plants  for  a  linear  system  will  be  treated  in  detail  later. 

We  now  formulate  a  new  performance  criterion  "V"  where 


T 


f (s;x(s) j  u ( s ) j  w ( s ) ) 


4-  t/(T,x(T)> 


where  ^(s)  is  the  Lagrange  multiplier. 


(1.6) 
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Note:  Finding  a  stationary  point  of  "V"  with  respect  to  the 

variations  in  u(s),x(s),  /^(s),  and  x(T)  unconstrained  is  the  same  as 

finding  a  stationary  point  of  "V”  with  respect  to  the  variations  x(s), 

u(s),  /^(s),  and  x(T)  subject  to  the  constraint  of  equation  (1.1). 

# 

We  now  find  the  stationary  point  of  "V"  with  respect  to  the 

variations  in  u(s),x(s),  /^(s)  ,  and  x(T) .  Carrying  out  the  variational 

problem  and  using  the  notation  of  Appendix  "A"  along  with  the 

variational  notation  (8,Chp,2)  we  get  for  stationary  "V"  (ie  &V  =0) 

T  f  ~ 

Lx(s;T,x(s)  ,u(s)  ,w(s))  &x(s)  +  LM(s;T,x(s)  ,u(s)  ,w(s))&u(s)  + 


T 

*£  L.  T  T 

A  (s)  dx(s)  __  f (s;x(s) ,u(s) ,w(s))|  +  A  (s) 


&  dx(s) 
ds 


ds  + 


fx(s;x(s) ,u(s) ,w(s))&x  _  fu(s;x(s) ,u(s) ,w(s))&u(s) 

T 

Lx(t;T,x(t)  ,u(t)  ,w(t))&x(t)  _  ^(t)fx(t;x(t)  ,u(t)  ,w(Jt))&x(t)  + 

l4(T,x(T))&x(T)  . (1.7) 

We  now  consider  the  term 
T, 

iT 


Jx 

\(s)&X 


( s )  &  dx(s)  ds 

ds 
T 


J&x  (  s  )  d  A 

ds 


and  integrating  by  parts  we  get  (1.8) 
(s)  ds  . ..(1.8) 


substitute  equation  (1.8)  back  into  the  equation  (1.7)  and  collecting 

terms  in  the  variations  of  &x(s),  &u(s)  ,  and  6c  ^(s)  also  take  the 

transpose  of  the  complete  equation  to  get  (1.9). 

T  r  T 


I 


&x(s) 


Lx(s;T,x(s) ,u(s) ,w(s))  _ fx  (s ;x(s) , u(s) ,w(s) )  ^(s)  _ 


d  A(s) 

ds 

+  &  A(») 


4-  &u(s)^ 

Lu(s;T,x(s) ,u(s) ,w(s))  _ 

_  fj(s;x(s),u 

dx(s) 

ds 

f  r(s;x(s)  ,u(s)  ,w(s))j 

ds  +  &x(t) 
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+  6cx(T) 


Lx(t;T,x(t) ,u(t) ,w(t))  —  ^(t) 

i/x(T,x(T)T  l/x(T,x(T))  +X(T) 


—  fx(t;x(t) ,u(t) ,w(t))  ^(t) 

. (1.9) 


All  the  variables  under  the  integral  are  not  constrained  and  therefore 

their  differentials  are  independent,  equating  their  coefficients  to 

zero  one  obtains  the  following  Euler  Lagrange  Equations 

dx(s)  =  f (s;x(s) ,u(s) ,w(s)) 
ds 


dAfs)  -  Lx(s;T,x(s) ,w(s)) — fx(s;x(s) ,u(s) ,w(s))  A(s) 
ds 

f J(s;x(s) ,u(s) ,w(s))  \  (s)  -  Lu(s;T,x(s),u(s),w(s)) 

for  all  t  <  s  <  T  . (1.10) 


The  Euler  Equations  are  the  basic  necessary  condition. 

We  now  consider  the  boundary  value  terms  (i.e.  the  last* 


terms  of  (1.9) 

&x(t)  j^Lx(t;T,x(t)  ,u(t)  ,w(t))  — AU) —  fx(t;x(t)  ,u(t)  ,w(t))  ^(t)j 

+  &x(T)  [l^x(T,x(T))  +  A.(T)]  . d.H) 

Since  x(t)  is  the  current  system  state  and  cannot  be  changed  by 
application  of  u(t)  it  follows  that  all  &x(t)  must  be  zero.  Similarly 

&x^(T)  (i  =  1,2,  . n)  are  not  all  independent  because  of  the 

constraint  of  equation  (1.4). 

Taking  the  differential  of  (1.4)  one  obtains  (1.12) 
dg(T , x(T) )  -  gx(T,x(T))&x(T)  *  0  . (1.12) 


if  T  is  fixed. 

If  gi(T,x(T) )  “0  (i  -  1,2, 
and  non-redundant  conditions  "g  " 

X 

n^xn  matrix  "gx"  and  a  remainder 


. ng)  are  mutually  consistent 

can  be  partitioned  into  a  non-simgular 


gx 


.  •  -  ; '  ’  , 

D I  ,ir  R 

■ 

/  •  V'  • 

;  ,r\Ss.  : 

V  ) 


/  3, , 


*  A 

The  corresponding  n  components  od  dx(T)  are  called  dx(T)  and  the 


remainder  denoted  by  d>c(T)  .  Thus  equation  (1.12)  will  become  (1.13) 
A  A  V  V 

gxdx(T)  +  gxdx(T)  -  0  . (1.13) 

A 

solving  for  dx(T)  we  get  (1.14) 


A 

dx(T) 


gx  *  §x  dx(T)  . (1.14) 


The  last  terms  of  (1.11)  becomes 


A(T)  +  yx(T,x(T))  —  g^  ‘  T[ACT)  +  y^(T,x(T))j  'dx(T)  . (1.15) 

A  A  J  ^ 

where  )  ^(T)  and  y  4  are  the  components  corresponding  to  &x(T)  whereas 

x1 


>M(T)  and  are  those  components  corresponding  to  &x(T)  . 

xx 


V 


V 


Now  exi(T)  (i  =  ng.-„^  »ng^2> 


,n)  are  independent  and  the 


vanishing  of  their  coefficients  requires  that 


V  T  A-!  TfC* 


ACT)  +  yv(T,x(T))  =  gx  gx 


AT)  +?yA(T,x(T)) 


(1.16) 


This  set  of  n-n^  conditions  on  A . (t)  or  x  (t)  (i  =1,2,  . ,n),  along 

o  — 

with  the  n  conditions  (1.4)  completely  specify  the  end  conditions. 

O 

1-  n  =n  equation  (1.4)  uniquely  specifies  x(t)  so  that  none  of  the 

O 


:x 


C,  (t)  (i 


—  y  ^  y 


,n)  are  independent. 


If  n  =0  x(l)  is  completely  free  and  then 

O 


A  (T)  +  U  (T,x(T))  =  0  . (1.17) 

These  are  the  so  called  natural  boundary  conditions. 

If  l^(T.xT)  is  independent  of  x(T),  then 

A(t)  =  0 
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Addi-  onal  Necessary  Conditions 

Additionsl  necessary  conditions  for  the  optimal  control 
problem  can  be  obtained  by  considering  the  classical  work  of 
Weierstrass  and  Legendre. 

Forming  the  well-known  Weierstrass  E-  function 


E(x,u,u  ,s)  =  L(x,u,s) 


L(x,u  ,s) 


/  o  ,  u  ,  \ 

-  \U  -  U  ,  Lu(x,u  ,s)> 


- (1.18) 

and  by  the  Weierstrass  necessary  condition  (see  30  it  must 
be  the  case  that  E(x,u,u°,s)  ^  o.  It  is  clear  by  inspection 
that  Eis  the  quadratic  remainder  in  the  Taylor  series  of  L 
at  u  =u°.  Using  the  well-known  estimate  for  the  remainder  we 
have 


E (x,u,u  ,s)  =  [ I u  —  u 


L  (x.u  +  8  (u 
uu  ’ 


o. 

U  ) 


S ) 


(1.19) 


where  fi  is  0  <  0  ^  1 

which  is  nonnegative  definite  if  L  >0.  This  is  just 
^  uu  x  J 

Legendre's  necessary  condition. 

Mote  :  <  is  the  well  known  inner  product  i.e.,  <.  x,y  J  — 
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SECTION  I 

PART  II  ADDITIONAL  NECESSARY  CONDITIONS  AND  SUFFICIENCY  CONDITIONS 

Another  approach  to  the  problem  is  to  use  Classical  Hamilton- 
Jacobi  Theory  as  applied  to  Calculus  of  Variations. 

Hamilton  and  Jacobi  recognized  that  the  relationship  of 
reducing  a  first  order  partial  differential  equations  into  a  system 
of  associated  ordinary  differential  equations  could  be  reversed.  In 
general  of  course  the  integration  of  a  partial  differential  equation 
is  usually  a  more  difficult  problem  than  that  of  solving  an  ordinary 
differential  equation.  In  mathematical  physics  and  mechanics  one  is 
often  led,  however,  to  a  system  of  ordinary  differential  equations 
in  canonical  form.  These  equations  may  be  difficult  to  integrate  by 
elementary  methods  while  the  corresponding  partial  differential 
equation  is  manageable. 

Eulers  differential  equations  in  canonical  form  can  be 
placed  in  a  suitable  form  by  use  of  Legendre  Transformation  and  its 
inverse  to  obtain  an  appropriate  partial  differential  equation. 

Let  J(x(s) ,£(s) ,u(s) ,s)  =  L(x(s) , u(s) , s)  . . (2.1) 

+  i  T  /  dx  -  f  ( s  ;  x,  u) 

A  (  ds 

The  resulting  Euler  equations  for  the  above  functional  are 

_dJ.  —  Jv  -  0  (i  1,2, . ,n)  . (2.2) 

ds  xi  i 

We  now  replace  the  variational  problem  by  an  equivalent 
canonical  variational  problem  which  leads  to  a  system  of  2n  canonical 
differential  equations.  For  this  purpose  we  introduce  the  moments 


. 

o 
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A  55  ^  and  assume  that  x  can  be  calculated  from  this  equation 
in  an  appropriate  domain  (ijX^u^s)  This  condition  is  true  if 

J.  1  ztl  0 

xj  x±  J 

We  now  introduce  the  inverse  function  ( A ,x,u.,s)  under  the  Legendre 
transformation  where  x»u,s  are  the  untransformed  parameters. 

From  the  Legendre  transformation  it  must  be  true  that 
=.  J.(x,x,u,s),  x  *  J-f  ('A  ,x,u,s) 

X 

and  . (2.3) 

J(x,x,u,s)  +  yi  (  \  ,x,u,s)  =<x,^> 

Differentiating  equation  (2.3)  with  respect  to  "x”  we  obtain 

Jx(x,x,u,s)  4-  x(  \  ,x,u,s)  =  0  . (2.4) 

From  which  we  immediately  obtain  the  following  canonical  form  for  the 
Euler  equations 

i  ■  ^4  ^  <1  •x>u>9)  . (2.5) 

^  ■  Jx  ‘  -TLA  A -x>u-s) 

In  turn  these  equations  must  satisfy  the  Hamilton  Jacobi 

equation 

VF  (Vx,u,x9s)  =  0  ... - ..............(2.6) 

Replacing  the  value  for  J(x„xj,u,s)  we  see  that  canonical  form 

becomes 

dc(s)  =  £(s;x(s),u(s)) 

ds  ..,..................(2.7) 

dX.  (s)  -  Lx(s;x(s) Su(s))  —  f x(s ; x(s) s u(s) )  ^  (s) 

ds 


Lx(s;x(s) s  u( s) ) 


-  v  J  <0 


.  ' 


'' 


■  ,v'-  '  ;  '  .  a  '  .  'j  . 

•  :  '"V  “  - 

■  ■  }  f  -  l  ‘  (  .  ;  ;  •. 

' 
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ictn 

and  the  Hamilton  becomes 

y<  (Vx,x,u,s)  =  -L(s ;x(s) , u(s) )  +  X(S)>  .....  .(2.8) 

If  we  let  p(s)  =  -  \  (s)  the  Hamilton  becomes  the  negative 

of  the  so-called  pre-Hamiltonian  in  Pontryagin's  Maximum  Principle 
(see  ). 

The  pre-Hamiltonian  is  given  by 

>Tlr(Vx,x,u,s)  =  L(x,u;s)  +  <  x,  £(s)  > 

Controllability  and  Observability 

The  concepts  of  controllability  and  observability  where 
developed  by  R.  E.  Kalman,  L.  Markus  and  E.  B.  Lee  with  the  majority 
of  the  work  contributed  by  R.  E.  Kalman.  These  concepts  are  in  their 
infancy  stages  and  a  great  deal  of  work  is  still  required  to  complete 
the  study,  as  yet  the  concepts  apply  only  to  linear  systems  except 
for  the  Local  Controllability  Theorem.  These  concepts  are  sufficiency 
conditions  and  for  this  reason  do  not  alone  determine  whether  or  not 
a  system  is  controllable  or  observable. 

Definition:  A  state  x(s)  is  said  to  be  controllable  at 

i 

time  tQ  if  there  exists  a  control  function  u  (s),  depending  on  x 
and  tQ  and  defined  over  some  finite  closed  interval  ^tQ,Tj  ,  such 
that  (p  2(T;x,tQ)  =  0  (for  terminal  point  at  the  origin).  If 

this  is  true  for  every  state  x,  we  say  that  the  plant  is  completely 
controllable  at  time  tQ;  if  this  is  true  for  every  tQ,  we  simply  say 
that  the  plant  is  completely  controllable. 


i 


This  definition  can  also  be  stated  in  terms  of  differential 
equationsas  follows. 

A  linear  system  (B-6,7)  is  completely  controllable  at  time 
tQ  if  it  is  not  algebraically  equivalent,  for  all  £s  tQj  ,  to  a 
system  of  the  type 


dx* 

ds 

=  F"  ( s  )  x 9 

+  F^(s)x^  +  G-(s)u(s) 

dx^ 

ds 

=  F^2(s)x2 

. . . . . . . . .  o . '  ( 2 , 9) 

y(s) 

=  H^(s)x^(s) 

+  H^(s)x^(s) 

where  the  vectors  x^-  and  x^  are  subgroups  of  the  state  vector  x  and 
contain  and  n2  =  (n  —  n^)  components  respectively, 

In  other  words,  it  is  not  possible  to  find  a  coordinate 
system  in  which  the  state  variables  x^  are  separated  into  two  groups, 
x*  =  (xi>  ,,sXnj^  anc*  x^  =  ^xnl*  •••»xn)>  suc^  that  the  second 
group  is  not  affected  either  by  the  first  group  or  by  the  inputs  to 
the  system.  If  one  could  find  such  a  coordinate  system,  we  would  have 
the  state  of  affairs  depicted  schematically  in  Fig.  (1-2.1) 

Controllability  is  a  system  property,  like  stability,  which 
is  completely  independent  of  the  way  in  which  the  outputs  of  the 
system  are  formed.  It  is  a  property  of  the  couple  ^F(s),G(s)j 

Definition:  Observable  Costates,-  A  vector  x#  in  the  dual 
space  is  called  a  costate.  The  value  of  the  real  linear  function  x# 
[«#.«]  • 


at  x  is 


' 


?:  '  ‘  -  1  ./  i  .  f  ! 

: 1 

83  ;  .1  en  it  ^y,.f  a,!;  arc:  ,:v 

•  ftii  x 

■  ' 

. 
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, 
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A  costate  x#Q  of  (B-6,7),  with  u(s)~  0S  is  observable  at 
time  tQ  if  there  exists  a  linear  function  u#(s)  defined  on  some 
interval  T_j,tl  and  depending  id  general  on  x#Q  and  tQ,  such  that 


(2.10) 


In  other  words,  a  costate  observable  at  tQ  is  a  linear  function 
whose  value  can  be  explicitly  evaluated  (no  matter  what  the  actual 
state  of  the  system  happens  to  be)  by  means  of  a  linear  operation  on 
the  output  of  the  system  during  a  finite  interval  preceding  tQ. 

A  state  xc  is  unobservable  at  time  t0  whenever  [x^,xQ  j  =  0 

for  all  x^. 


We  can  also  define  observability  in  a  slightly  different 

manner  as  follows.  A  linear  dynamical  system  (B-6,7)  is  completely 

observable  at  time  tQ  if  it  is  not  algebraically  equivalent,  for  all 

s  =  tD,  to  any  system  of  the  type 

dx*  =  F 11  (s)x*  +  G  (s)u(s) 

ds 

dx2  "  F2,(s)x*  +  F^(s)x2  +  G2(s)u(s) 

ds 


y (s)  =  H 1  ( s ) x  1  (s) 

This  state  of  affairs,  if  it  existed, 


(2.11) 


is  depicted  in 


Fig.  (1-2.2). 
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Motivation  for  Controllability 

Consider  the  problem  where  the  loss  function  L(x,u,s)  is 


given  by  1/2  ||  u(s)  ||  ^ .  This  problem  is  sometimes  called  the 

minimum  energy  problem  because  the  integral  of  this  particular  loss 

function  is  a  measure  of  energy. 

The  resulting  Euler  equations  are 

dx  **  F(s)x(s)  +  G(s)u(s) 
ds 

dA  -  -FT(s)A(s)  . (2.12) 

ds 

u(s)  ■  Gt(s)  A(s) 

The  solution  for  ^(s)  is  given  by 

A<s;£0*co>  -  A  c s,ta )  . (2.i3) 


where  Vj/  ( s , t0)  satisfy 

dA  -  -FT(s)  (s,t0)  . (2.14) 

ds 

From  Appendix  (B)  we  see  that 

A(s.t0)  -  $  T(t0,s)  . (2.15) 


Using  these  results  the  control  law  now  becomes 
u  ( s)  =  GT<£T(t0,s) 

We  have  yet  to  establish  a  relationship  between  ^  0  and  xQ.  To  do 
this  consider  the  solution  for  the  system  using  equation  (B-10). 
(p(s;x0,t0)  «  (s,t0)x0  +  T(5 (s,CT)G(CT)GT(Cr)  ^  T(tOJ^) 


Define 


W(tQ,s)  - 


(t0,cT)GT(Cr)  $  T(t0,£T  >d(T 


(2.16) 


, 


\B  )  0 

f  n  .  1;  v. (  :)  .(  *o$  in:  ■  n< '  os 

„,v„  '  4' 


C,  ;;  '* )  /\  ■  ■'  ,:t 


a  ■  [n  )  n.l  vn  i.r,:  ■' 

-C  •  ■■  -  ..  ■  .1.1  . 
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2 


and  evaluate  equation  (2.16)  at  s  =  T. 

<|>  (T;x0,t0)  =  (£>(T3t0)x0  +  (J>  (T,t0)W(t0,T)  ......  . 

Solving  equation  (2.17)  for  to  get 

^(t0,T)x(T)  —  xQ  =  W(t0,T)  £0  .......  ..... 

Using  this  result  we  now  see  the  control  law  becomes 

u(s)  =  -GT(s)  ^T(t0Ss)W-1(t0,T)  [  xQ  —  (|>  (t0,T)x(T) 


.(2,17) 


(2.18) 


(2.19) 


From  which  we  find  that 
T, 


Jl|u(s)  II 


'ds 


‘o  —  C^(t0,T)x(T) 


W  (t0,T) 


From  which  we  see  that  for  the  energy  to  be  positive  (which  it  must  be 
if  the  system  is  to  have  physical  meaning)  it  must  be  the  case  that 
W(t0,T)  be  positive  definite.  The  extension  of  this  idea  to  the  general 
case  is  done  by  hypothesis,  that  is  to  say  we  assume  that  the  above 
condition  holds  true  in  general  and  proof  that  this  is  the  case. 

To  determine  controllability  of  a  system  the  following  lemma 
can  be  used. 

Lemma  1:  A  plant  is  completely  controllable  at  time  s(i) 
if  and  (ii)  only  if  the  symmetric  matrix 

T, 


W(t  ,T)  = 


J(^>( t0,s)G(s)GT(s)  C^>T(t0s)ds 


. (2.20) 


is  positive  definite  for  some  T^>  tQ„ 

An  equivalent  statement  is  that  W(t0sT)  must  satisfy  the 
following  differential  equation. 


dW(t0,T) 

dt„ 


F(t0)W  +  WFT(tQ)  —  G(t0)GT(t0) 


■  t  1  •  j  -,r ,  '  v.,  /; 

<0**0 


■  1  :n  rvr  ’  r:  *  "  j  ?t  ; 


' 
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subject  to  the  condition  that  W(T)  =  0 

Proof:  (i)  set  u1  (s)  =  -GT(t)  <^j)  T(tQ,  s)^1  (t0#T)x(t0) 

...  o  .....  o  o (2.21) 

and  then  substitute  this  into  the  solution  of  the  plant  given  by  equations 
(B-6,7)  with  a  solution  given  by  equation  (B-10).  It  is  desirable  to 
return  some  system  to  the  origin  by  a  control  function  u1  (t)  subject 
to  some  performance  criterion,  Hence,  if  upon  substitution  of  u'  (s) 
into  the  solution  of  the  plant  equation  the  state  is  returned  to  the 
origin  the  Lemma  is  proved  for  (i).  Upon  substitution 

(pu,(T;x0,to)  =  ^  (T,t0)x0  —  J$(T^)G(£)Gt(^)$t 

(to»f  >W’(to»T)  *<to)<*f  .........(2.22) 

By  the  lemma  and  from  the  fact  that  $(3  »£)  =  <f>  (s^o)  <§(to>£) 

equation  (2.22)  becomes 

(j^T;x0,to)  -  $  (T.t0)x0  —  5>(T,t0)x0 

Hence  ^,(T;x0,t0)  =  0 

(ii)  We  will  prove  this  part  by  contradiction.  By  the  lemma  it  is 

required  that  the  matrix  W(tQ,T)  be  positive  definite. 

Assume  that  there  exists  an  x  =£  0  such  that  II  x  II  2  =  0 

W(t0sT) 

Define : 

u2(s)  =  -GT(s)  <J>  T(toSs)x(t0) 

which  implies  that 


W(t0,T) 


2 


0 


t  no  '  • 


3V  •  . 


■?t  1  VT  •  :  ijt  91'i  u  ■:  ■■ 


J  ~  0  '  v 


Since  u^(s)  is  continuous  in  s,  it  therefore  identically  zero  in  the 
interval  j  tQ,T  1  . 


On  the  other  hand,  if  the  plant  is  completely  controllable  at 
tQ,  there  exists  a  control  function  u*  (s)  from  (i)  which  satisfies 
the  relation 


x(tQ)  = 


t  j^0’ 


s)G(s) u ' (s) ds 


(2.23) 


To  see  that  this  relationship  holds  just  substitute  the  value  for  ux(s) 
given  by  equation  (2.21)  into  equation  (2.23)  to  get  the  result 
x(tQ)  =  x(tQ). 

Therefore  it  must  be  the  case  that 


contradicting  the  assumption  that  x(tQ)ji£:  0  Q.E.D. 

The  last  part  is  proved  by  differentiating  equation  (2.20)  with 
respect  to  "t0". 


T 

dW(t0,T)  -  fi>_  J(t0,s)  [G(s)GT(s)^T(t0>s)] 

dtQ  t0^  dto  J- 

T 

f<5( t0,s)G(s)GT(s)  1—  (J  T(tOJs)ds  -G(t0)GT(t0) 
t0->  X  Oto 

and  from  the  equations  d^(tQ,s)  =  F(tQ)  (j}(t05s) 

dtQ 

^  T(tn,s)  =  (^)  T(t0,s)FT(t0)  we  get 


ds 


and  d 
dt. 


dW(tQ,T)  =  F(t0)W(t0,T)  +  W(t0,T)FT(t0) 

dtQ 


G(t0)GT(t0) 


+ 

..(2.24) 


and  this  completes  the  proof. 


' 
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■rile  proofs  given  for  controllabilty  are  applicable  when 
the  terminal  point  of  the  system  is  the  origin.  It  follows  easily  by 
a  slight  extension  of  the  preceding  arguments  that  z(s)  (where  z(s) 
is  some  desired  state  at  s  =  l)  is  reachable  from  x(i.e.,  there 
exists  a  motion  ^  which  meets  x  at  t  and  z  at  T)  if  and  only  if 


the  equation 

x(tQ)  -^)(to,T)z  =  W(to,T)v(tQ)  . (2.25) 

has  a  solution,  in  which  case 

uX(s)  =  -Ga(s)  $  T(to,s)v(tQ)  . (2.26) 

is  the  appropriate  control  function. 


To  show  this  substitute  u^(s)  into  equation  (B-10)  to  get 


o 

(to?)v(xo)df 

which  becomes  from  lemma  (i). 

CPu.' (TiXo,to)  =  $(T,t0)x0  -  W(to,r)v(to)  . (2.27) 

Assuming  from  the  lemma  that  W(t  ,T)  is  positive  definite  we  can  solve 
for  v(t  )  from  equation  (2.25)  to  get 

v(tQ)  =  W_1(to,T)  £  x(tQ)  “  ^(tQ,T)z(T)J 

which  in  turn  is  substituted  into  equation  (2.27). 


nence 


=  #d.to)xo  -  w(to>T)w-1(to,T)[xo  -  9(t0,T)2] 

=  (|> (t  ,T)z(T) 
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The  above  conditions  are  applicable  for  linear  time  varying 
systems,  there  is  a  somewhat  more  convenient  form  for  constant  linear 
systems.  In  the  case  of  constant  systems  the  notions  of  complete 
controllability  and  complete  observability  do  not  depend  on  the 
choice  of  tQ.  Lemma  (2):  A  constant  plant  is  completely  controllable 
(i)  if  and  (ii)  only  if 

rank  |  G,FG,  . , Fn-IG  ]  -  n  . (2.28) 

Proof:  (i)  By  the  previous  lemma  it  is  sufficient  to  show  that 

W(0ST)  is  positive  definite  no  matter  how  small  T  —  tQ^>0. 

Let  g'  . ,gm  be  the  columns  of  G,  If  W(0,T)  is  positive 

semidef inite,  then  proceeding  as  in  part  (ii)  of  the  previous  lemma 


we  conclude  that  there  is  a  vector  x.(0):=f=  0  such  that  jj  x|| 


^  =  0. 
W(0,T) 


Define 

u2(s)  =  -gt 

which  implies  that 


X 

2 

r 

[ u^(s)  1  ^  ds  = 

0 

W(0,T) 

0  J 

L  J 

and  since  u^(s)  is  continuous  in  s,  it  is  identically  zero  in  the 
interval  ^0,T  J  . 

Therefore  in  the  case  of  constant  coefficients. 


u^(s)  =  -GTexpFTs  x  =  0  ..................  (2 . 29) 

and  therefore, 

£x,  expFs  gi  ]  -  0  for  i  =  I,  . ,m 

1 

where  (g  ,  g2v  •  •  •  gm)  are  the  columns  of  G 


•• 


3-c  t&  ai  Buo*>ai inoo  ax  (a  )~u  von 
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Differentiating  j  times  with  respect  to  s,  and  then  setting  s  =  0,  we 

Set  r  .  .  i 

x,  FJ  g1  j  =  0  for  all  i  =  I„  ......  ,m 

and  j  =  I, ...... 9n-I 


. . . (2.30) 


If  equation  (2.28)  holds,  this  implies  that  x  is  orthogonal  to  set  of 
generators  of  En,  contradicting  the  assumption  that  x^-,0, 

(ii)  Assume  the  plant  is  completely  controllable  but  (2,28) 
is  false.  Then  there  is  a  vector  x  ^-0  which  satisfies  (2.29),  By  the 
Cayley  -  Hamilton  theorem 

/j:)  g1]-  [*, 

where  i  *  I . . m 


x,  expFs  g: 


x, 


£  (Fs) J 


j=o 


It  follows  that 


W(03T) 


■  0  for  all  T,  contradicting  the  assumption 


of  complete  controllability.  Q.E.D, 

To  determine  observability  the  following  lemma  is  used. 
Lemma  3:  A  plant  is  completely  observable  (i)  if  and 

(ii)  only  if  the  symmetric  matrix 


M(to3T_T)  = 


to 


T-I 


JCpT(s,t0)HT(s)H(s)  ^(s,  tQ)  ds 


(2.31) 


is  positive  definite  for  all  tQ. 

This  idea  can  also  be  extended  to  linear  constant  systems 
under  the  following  lemma. 

Lemma  4:  A  constant  plant  is  completely  observable  (i)  if 
and  (ii)  only  if 

rank  [ HT , FTHT (FT) n- IHT  ]  =  n 


serf 3  wwof.r  j'i  3.r 
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Proof:  The  proof  of  these  lemmas  is  achieved  indirectly  by 
showing  that  all  theorems  on  controllability  can  be  "dualized"  to 
yield  analogous  results  on  observability. 

Duality  relations 

(i)  s  -  t0  -  tG  -  t 

(ii)  F*(t)  =  FT( s)  ,  G*(t)  =  HT(s)  ,  H*(t)  =  GT(s) 

( iii)  $*(c»t0)  =  C|>  T(t0,s) 

where  the  dual  system  is  given  by 

dx*  -  F*(t)x*  +  G*(t)u*(t) 

dt 

y*(t)  »  H*(t)x*(t) 

Hence  it  can  be  stated  that  a  plant  is  completely  controllable 
if,  and  only  if,  its  dual  is  completely  observable,  and  conversely. 
Definition:  A  plant  is  uniformly  completely  controllable  if  the 
following  relations  hold  for  all  s: 

(i)  0  <  0(  ((f)  I  2  W(s,s  +(j)  =  CX,  (CT)I 

o 

(ii)  o  <  Bc  (CT)I  =  (s  +CT,s)W(s,s  +CT)$T(s  +cTss)  ^  B  ((^)I 
where  CT  is  a  fixed  constant.  In  other  words,  one  can  always  transfer 

x  to  0  and  0  to  X  in  a  finite  length  (j  of  time;  moreover,  such  a 
transfer  can  never  take  place  using  an  arbitrarily  small  amount  (or 
requiring  an  arbitrarily  large  amount)  of  control  energy. 

Similarly  we  can  define  uniformly  complete  observability  by 
use  of  the  duality  relations.  (i.e.,  if  a.  plant  is  uniformly  com¬ 
pletely  controllable  its  dual  is  uniformly  completely  observable). 


. 
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The  conditions  of  uniform  complete  controllability  and 

observability  will  be  used  in  a  later  section  to  prove  a  stability 

theorem  for  a  linear  system. 

By  using  the  Gronwall- Bellman  lemma: 
t2r  |, 

F(T>||  dT  s  7  C^2  “  fcl)  for  a11  tl»  and  t2 


We  arrive  at  the  condition  that 

||<|(s.r)||  =  «3(  I  »  — r|  )  for  all  s. T  . (2.32) 

The  ideas  of  controllability  is  extended  to  the  nonlinear 


equation 

dx  “  f(s;x,u)  where  f(s;0,0)  =  0  . (2.33) 

ds 

Linearization  of  f(s;x,u,)  results  in  the  equation  (B-6,7)  where 

F(s)  =  c)fi(s;0t0)  and  G(s)  =  d  f d(s ;0.0) 

d  xj 

Local  Controllability  Theorem 

The  nonlinear  equation  (2.33)  is  locally  completely 
controllable  at  the  origin  at  time  tQ  if  (B-6,7)  is  completely 
controllable  at  time  tQ. 

Proof:  Assume  that  a  solution  for  equation  (2.33)  exists, 
(Note,  it  can  be  shown  that  under  certain  conditions  a  solution  to 
(2.33)  does  exist  but  this  is  beyond  the  scope  of  this  paper.). 


We  let  u^  be  the  function 

u(s,£)  -  +  GT(s)<^T(t0,s)f  . (2.34) 

From  the  solution  to  the  equation  (B-6)  we  see  that, 

<p(T;x0,t0,uj  )  “  cj)  (T,t0)x0  +  <|)  (T,t0)W(t0.T)  . (2.35) 
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so  the  Jacobian  is 


[d<P*(TiX0,to,U£)/  j]  >  J  (T,to)W(t0; 


T) 


which  is  nonsingular. 

Hence  by  implicit  function  theorem  we  can  solve  for  t 
term  of  x0,  denote  as 

u£  =  m  (xq) 

Further  it  must  be  the  case  that  (p  (T;x0Jt0u)  =  0 


which 


is  the  requirement  that  the  plant  must  obey. 


■  ■  ;.n .  .  .... 
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SECTION  II 


PART  I  SOLUTION  OF  THE  TWO  POINT  BOUNDARY  VALUE  PROBLEM  FOR  A 
~  CONSTANT  COEFFICIENT  LINEAR  SYSTEM 

In  the  case  of  constant  coefficient  linear  systems  the  plant 

can  be  described  by  the  following  differential  equations 

dx(  s)  =  Fx(s)  +  Gu(s)  +  Kw(s)  . . (1.1) 

ds 

where  F  is  a  constant  n  x  n  matrix 

G  is  a  constant  n  x  m  matrix 

K  is  a  constant  n  x  tr  matrix 

The  output  of  the  system  is  expressed  by  the  equation 

y (s)  »  Hx(s)  . (1.2) 

The  problem  of  a  linear  system  with  constant  coefficients  has 
been  considered  in  detail  by  methods  other  than  presented  here,  most 
of  these  methods  employ  Laplace  transforms  and  Wiener's  Spectorial 
Factorization  techniques. 

The  performance  criterion  to  be  considered  is  of  the  form 

Tf 

V  =  Lim.  /  L(x(s)  ,  u(s) )  ds  . (1.3) 

T  - c»  oo  t 

subject  to  the  general  end  point  boundary  conditions  of  the  form 

Ax(T)  +  B  ^  (T)  =  d  . (1.4) 

where  A  and  B  are  constant  matrices. 

Further  we  assume  that  the  loss  function  L(x(s),u(s))  is  a 
quadratic  function  of  x(s),u(s),  and  z(s)  with  constant  coefficients 
and  T  is  fixed. 

The  desired  output  of  the  system  is  denoted  by  z(s)  and  is 


included  in  the  loss  function. 
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Let  us  consider  the  quadratic  loss  function  of  the  form 

L(x(s) ,u(s) )  =  if  ||y(s)  —  z(s )i(  +  |Ju(s)ll  \  } 

L  Q  R  J 

. .  (1.5) 

upon  substitution  of  equation  (1.2)  into  equation  (1.5)  we  get  the 
loss  function 

L(x(s),u(s))  =  i  £l(  Hx(s)  -  z(s)||^  +  |)u(s)||  ^ 

.  (1.6) 

where  Q  and  R  are  symmetric  positive  definite  constant  matrices. 

With  the  above  restrictions  imposed  on  the  system  the  Euler 
equations  become 

dx(s)  =  Fx(s)  +  Gu(s)  +  Kw(s) 
ds 

d  A  (s)  =  H^QRx(s)  -  F^  A  (s)  -  H^Qz(s) 

ds 

Ru(s)  =  G^  A  (s)  . . . .  (1.7) 

because  R  is  nonsingular  we  can  solve  for  u(s)  to  get  the  linear 
control  law 

u(s)  =  R  G^  A  (s)  .  (1.8) 

Now  substitution  of  equation  (1.8)  back  into  the  Euler  equations 
results  in 

-1  T 

dx( s)  “  Fx(s)  +  GR  G  A(s)  +  Kw(s) 
ds 

d  A  (s)  -  HTQKx(s)  -  FT  A  (s)  -  HTQz(s)  . (1.9) 

d  S 
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rewriting  the  abovf  equations  in  the  following  form 


dx(s) 

F  GR'1GT 

x(s) 

Kw(s) 

dsi 

d  A  (s) 

ds 

T  T 

d  QH  -F 

A(s) 

+ 

T 

-H  Qz(s) 

(1.10) 


and  diagonalize  the  matrix 


formation  U  such  that 


j? 

T 

H  QH 


-F 


under  the  unitary  trans- 


F 

T 

H  QH 


GR' LGt 
T 
-F 


,-l 


InT 


If  we  investigate  the  constant  matrix  /  F  GR  G 

T  T 

H  QH  -F 


we  find  that  the  right  half  of  the  "s"  plane  (Note;  s  here  is  the 

Laplace  transform)  has  "n"  roots  whose  real  part  has  an  opposite 

sign  in  the  left-hand  side  of  the  "s"  plane.  Using  this  fact  that 

the  roots  are  symmetrically  located  about  the  imaginary  axis  in 

the  s  plane  we  subdivide  the  diagonalized  matrix  D  into  two  parts 

Dr  for  roots  with  positive  real  parts  and  for  roots  with  negative 

real  parts.  Further  it  must  be  the  case  that  Dr  *»  -D^  . 

In  cases  of  roots  on  the  imaginary  axis  we  assume  that 

they  have  a  real  part  of  . 

The  fundamental  solution  for  Dr  and  will  be  designated 

as  and  <£>r(t,t0).  They  must  in  turn  obey  the  equations 

d<f>i(t,t0)  -  D1(J)1(t,t0) 

dt 

d  r(t>to)  “  Dr$r(t»to) 

dt 


) 
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where  ^i(t,t0)  decays  to  zero,  where  Cj3r(t,t0)  grows  unbounded 


as 


O c> 


Under  the  transformation  of  the  matrix  U  to  diagonalize 
the  matrix 


GR  GJ 


we  now  define  a  new  set  of  variables 


lQH 


f£<s>' 

i7?(s), 


=  u 


x(s) 

A(s> 


Thus  equation  (1.10)  becomes  upon  substitution  of  the  new  variables 

/  F  GRGT\  /  f  (s)\  Kw(s)  \ 


\  htqh  -ft 


>?(.)/ 


y  -htqz(s> / 


we  now  multiply  both  sides  by  U  and  using  the  fact  that  UU”^  =  I 
we  get 

L 


d£U) 

ds 

ds 


Dr  o 


D  D, 


\ 

/w(s)  \ 

^(o)  / 

+ 

u 

\w(s) 

/s  V  T 

where  w(s)  «  Kw(s),  and  w(s)  =  -H  Qz(s). 

The  solution  of  these  equations  at  s  »  T  is  given  by 

f(T)  =  $r(T,t)£(t)  +  J$r<T,tf)w*«j)dCJ' . (l.U) 

T}m  -  ^(T.t)  7?(t)  + 


In  terms  of  the  new  variables,  and  partitioning  U  into 


U11  U12 


inverse  is  given  by 


V11  V12\ 


U21  U22 


whose 


1V21  V22 I 


Jthe  boundary  conditions  become 


.  , 


y  .i’-'f'iiu)  Wi'Jfi':  *)'(■? 


0  rz 


1  '-3f 


r*  \ 


i  V  «.  :J  ,  :  (• ' 


■  '  •"  'i  1  Jo  'P  .. 


35. 


<AVU  +  BV21)  £(T)  +  (AV12  +  BV22)  7J(T)  -  d  . . . (1.12) 

substitution  of  equation  (1.11)  into  equation  (1.12)  to  get 


(AVU  +  BV2l)  <j)r(T,t)£(t)  +  (AV12  +  BV22)^  X(T,t)  7J(t) 

1 1§  ,(T.^*(cr) dcr  +  J$lC t  ,0')«*(0')dCj'-  d  . . (1.13) 


Further  returning  to  the  original  variables  i.e.t 

and  equation  (1.13)  becomes 


£(s)  \  f  Un  U19  \  /  x(s)1 


7?(s) 


'11  12 
U21  u22 


(AV 


11  +  BV2i)^r(T.t)  [  Un  x(t)  +  U21  AU>  ]  +  (AV12  +  BV22j 


$l(T.t)  [u2lx(0  +  u22A.(t)] 
t  jf<|i<T.d>»*(tf><id' 


t  j^r(T>o')»*(0')dC' 


(1.14) 


If  we  assume  that  the  system  is  unforced  and  that  we  solve  the  regulator 
problem  with  the  final  state  of  y(s)  »  0,  Then  the  equation  (1.14) 
becomes 

[(AVU  +  BV21)^r(T,t)Uu  +  (AV12  +  BV22)  ^>1(T,t)U21  J  x(t) 

[(AV1X  +  BV21)^r(T,t)U21  +  (AV12  +  BV22)  <j}  1(T,t)U22  ]  \(t)  -  d 
Thus  from  the  control  law  we  get 

u(t)  ■  R"1GT,\(t)  where  ^(t)  is  given  by  the  following  equation 

^(t)  -  [(AVU  +  BV21)^r(T,t)U21  +  (AV12  +  BV22)^  1(T,t)U22  ] -1 

[d  ~  [(AVU  +  BV2i)(|>r(T,t)U11  +  (AV12  +  BV22)  X(T,  t)U21  ]  x(t)] 


' 
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In  the  steady  state  case  when  T— *■  o-»  the  equations  reduce  to 
u(t)  =  R'1GTU12"1U11  x(t)  . . .  (1 . 15) 

Stability  Properties  of  Euler  Equations  for  the  Linear  System 

The  symmetric  properties  of  the  matrices  in  the  Euler  equations 
imply  the  instability  of  these  equations.  It  will  be  shown  that  no 
stable  equilibrium  point  exists  and  that  every  equilibrium  point  is 
a  saddle  type  singularity  to  which  there  converge  "n"  stable 
solutions  and  to  which  there  diverge  "n"  other  unstable  solutions. 

Consider  the  following  Euler  equations 


H^QH  -  F 


gr-1gt 

T 


(1.16) 


let  R^-  GR  Gt  and  -  HTQH  results  in 


R# 


Q#  -FT 


x(s) 

U<«> 


dx(s)  —  Fx(s)  *  R#A.(s) 
or  ds  .,.,(1.17) 

d  A(s)  +  FtA.(s)  =  Q#x(s) 
ds 


Taking  the  Laplace  transform  of  the  above  equations  we  obtain 
(si  _  F)X(s)  —  R#A(s)  =  const. 

(si  F)a(s)  -  Q#X  (s)  =  const, 

rewrite  as 


(1.18) 


(F  —  sI)X(s)  +  R#A(s)  =  const. 

Q#  X(s)  -  (FT  +  sI)a(s)  =  const, 
where  the  const,  takes  care  of  the  initial  conditions. 


(1.19) 


* 


■  f 
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The  stability  of  the  equation  will  depend  on  the  roots  of  the 
polynomial  in  the  complex  "s"  plane 

F— si  R# 

Q#  -FT  -  si 

where  roots  in  the  right-hand  side  correspond  to  unstable  characteristic 
solution  (growing  exponentials),  and  roots  in  the  left-hand  plane 
correspond  to  stable  characteristic  solution  (decaying  exponential). 

Since  the  determinant  of  a  matrix  equals  the  determinant 
of  its  transpose 
F  -  si  R .# 

Q#  -FT  -si 

where  use  has  been  made  of  the  fact  that  Q#  and  R#  are  symmetric. 


By  interchanging  columns,  then  rows  one  obtains  successively 


l 

CO 

M 

JO 

-Q0  -PT  +  si 

F  +  si  R# 

R#  -F  -si 

-F  -si  R# 

Q#  -FT  +  si 

d  that 

FT  -  si  Q# 
R#  -F-sI 


(1.20) 


F  -  si 

R# 

M 

1 

CD 

R# 

Q# 

-FT  -  si 

ss 

Q# 

-FT  -  I(-s) 

..(1.21) 


Then*,  the  roots  must  be  located  symmetrically  about  the  imaginary 
axis  in  the  "s"  plane. 
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SECTION  II 


PART  II  SOLUTION  OF  THE  TWO  POINT  BOUNDARY  VALUE  PROBLEM  FOR  LINEAR 
NON- CONSTANT  SYSTEM 


In  the  case  where  the  coefficients  are  non-constant  (i.e., 
function  of  "s")  we  can  no  longer  employ  the  diagonalization  technique 
used  for  the  solution  of  the  constant  coefficient  linear  system.  We 
can,  however,  develop  a  general  method  which  can  be  solved  analytically 
in  some  cases  and  by  numerical  techniques  in  general. 

Consider  the  plant 

dx(s)  «  F(s)  x(s)  +  G(s)  u(s)  +  K(s)  w(s) 
ds 


y(s)  -  H(s)  x(s) 
subject  to  the  boundary  conditions 
x(s)  «  x(s) 

A  x(T)  +  B^(T)  =  d 


(2.1) 


(2.2) 


Once  again  we  let  the  loss  function  be  quadratic  and  of  the 


form 

L(x(s)  , u(s) )  -  l|fr(s)T  -  z(s)T)  Q(s)  (  y(s)  -  z(s))  +  1  u(s)T 

2  2 

R(s)  u(s)] . . . (2.3) 

where  Q(s)  and  R(s)  are  symmetric  positive  definite  matries  of  class 

C^  and  z(s)  is  the  desired  output. 

Forming  the  Euler  equations  and  eliminating  u(s)  we  obtain  a 

canonical  system  of  the  form 

dx(s)  =  F(s)  x(s)  -I-  R7'  (s)  ^(s)  +  w(s) 
ds 

d  A(s)  =  Q^(s)  x ( s )  — FT(s)  ^(s)  +  w(s) 

ds 


(2.4) 
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where  w(s)  =  K(s)  w(s) 

w(s)  =  -H(s)^Q(s)  z(s) 

and  R#(s)  =  G(s)R(s)G(s)T 

Q#(s)  =  H(s)TQ(s)H(s) 

Rewriting  the  equations  in  the  following  form 


(2.5) 


(2.6) 


4.x  (a) 

ds 

d-ks), 

ds 


F(s) 


R^(s) 


Q#(s)  -FT(s)  - 


x(s) 


A(s) 


w(s) 


V 

w(s) 


(2.7) 


R#(s) 


<B> 


Let  d)  (s,t)  be  the  transition  matrix  for  the  above  system 

and  denote  the  matrix  /  F(s) 

Q#(s)  -Ft(s) 

Thus  the  transition  matrix  must  obey  the  differential  equation 

"5$  (s.t)  -  <S>  («)  5)  (s,t) 

'Qs  x  . (2.8) 

we  now  partition  the  matrix  (s,t)  into  the  following  matrix 

$u(s,t)  $12(s.t) 

)  $  21 (s » fc)  $  22(s»fc^ 

using  this  fact  and  because  the  solution  must  also  exist  at  s  =  T  we 

can  form  the  solutions 

*r  A 

x(T)  =  ^u(T,t)  x(t)  +  $12(T,t)A(t)  +  0(T,t) 


^  (s,t) 


(2.9) 


\(T)  -  ^  21^t»  fc)  x(b)  +  $22(T,t)  A(t)  +  Q(T,t) 


....(2.10) 


where 


A 

Q(T,t)  \ 

^Q(T,t) 


T  /w(£)\ 

Ji'M. <f,)d£ 


(2.11) 
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using  this  solution  we  can  rewrite  the  boundary  conditions  as 
(A<^12(T,t)  +  B<J22(T,t))A.(t)  ‘  d-iO(T,t)-lQ(I,t) 

-  (A^n(T,t)  +  B<J21(T,t))  *(t)  . (2.12) 

Here  again  the  control  law  will  be  given  by 

u(t)  -  R(t)  G(t)TA(t)  . . . (2.13) 

where  ^(0  is  given  by  the  equation  (2.12). 

To  solve  the  control  problem  we  need  to  calculate  the  matries 
$(T.t),0(T,t)  and  0  (T,t),  These  matries  can  be  calculated  in  the 
following  manner, 

(£)(T,t)  can  be  calculated  by  solving  the  equation 
d  ^  (s,t)  ■  <8>  (s)  (s,t) 


or  rewriting 


2d 


£(©  (s))  11  ,t))ij  . (2.14) 

e--» 


where  t  If  s  T  and  i  =  (1,2, 

j  =  (1,2, 

subject  to  the  initial  equations 


,2n) 

,2n) 


(2.15) 


Further  the  equations  for  terms  Q  (s,t)  and  Q  (s,t)  can  be  formed 

A 

(s,t) 

(s,t) 


as  follows,  let  ^  (s,t) 
Thus 


from  equation  (2.11)  we  form  the  2n  integral  equa 

s,r  .  I  \  .  I  Ocs.t)' 

(s  J-f  -  if1""  ■(«<..«) 


quations 


(2.16) 
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We  can  reduce  this  to  a  system  of  differential  equations  of  the  form 

A. 


/  ^s> 

^CP(stt)  -  O  (s)  M  (s,t)  +  I 

bs  Y  \w(8) 


(2.17) 


for  s  Z5T 

Once  we  have  generated  (s,t)  and  (s,t)  for  t^stfT 
it  is  possible  to  obtain  more  than  a  solution  to  just  a  single 
boundary  value  problem.  It  may  well  be  that  one  wishes  to  solve  the 
problem  for  the  case  when  T — »  <x>  ,  It  is  then  of  interest  to  solve  it 
for  a  fixed  value  t  and  observe  how  the  coefficients  in  the  control  law 
behave  when  T — »oo.  This  is  known  as  the  floating  time  to  go  problem. 

In  a  different  approach  the  termination  time  T  may  be  fixed 
and  the  control  law  may  be  required  for  successive  value  of  t  as  t — s-  T. 
In  this  case  one  requires  (T,t)  and  {J5  (T,t)  on  some  interval 

tQ^t;fT.  To  obtain  this  one  generates  ^  (t,T)  by  solving  the 
equations 

2 1 


-r  ^n  a 

-  y  (©  (t)>  u  (t ,T)) x j  . (2. 

t 


18) 


1=1 


where  i  ~  =  (1,2, 

j  =  (1,2, 


,2n) 

,2n) 


subject  to  the  initial  condition  ^  (T,T)  =6i 


In  effect  we  are  solving  the  problem  by  what  is  called  the  shrinking 
time  to  go.  Thus  we  can  express  x(t)  and^(t)  as 
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SECTION  II 


PART  III  SOLUTION  OF  THE  SHRINKING  TIME  TO  GO  PROBLEM 

There  exists  a  somewhat  more  elegant  method  for  solving  the 
shrinking  time  to  go  problem  for  linear  time  varying  systems.  The 
method  which  is  to  follow  was  first  recognized  by  Kalman,  who  used 
Caratheodory  fundamental  equations  and  Hamilton  Jacobi  theory  in  its 
development.  The  same  results  are  developed  for  the  problem  posed 
by  Kalman  using  different  techniques,  calculus  of  variations,  and 
then  these  results  are  extended  to  solve  the  more  general  non- 
homogeneous  problem.  This  is  done  by  an  appropriate  transformation. 

In  the  problem  treated  it  has  been  shown  that  the  optimal 

control  law  is  linear  and  is  given  by  the  equation  (3.1) 

u(s)  =  C(s,T)x(s)  . . . . . ....(3.1) 

The  method  presented  formulates  a  way  of  solving  for  the  coefficient 

C(s,T)  directly  by  use  of  a  nonlinear  differential  equation  of  the 

Riccati  type  treated  as  an  initial  value  problem.  Further  it  is 

natural 

required  in  Kalman’s  problem  that^boundary  conditions  exist  at 
s  =  T.  In  situations  in  which  it  is  applicable,  this  method  yields 
an  excellent  way  of  solving  control  problems.  It  is  also  very 
interesting  to  note,  and  which  will  be  shown  later,  that  the  simple 
regulator  problem  solved  by  this  method  is  the  dual  to  the  linear 
time  varying  filter  problem. 

The  problem  to  be  solved  assumed  initially  that  the  desired 
equilibrium  state  of  the  system  is  zero,  we  then  generalize  the 
problem  to  the  case  where  the  system  is  to  follow  as  closely  as 
possible  some  desired  output. 
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The  performance  criterion  is  given  by 


4  a . 


V(s ,TS  x(s) ,  u(s) )  =  I/CxCT)^)  +  J  L(x(s),u(s)ss) 


ds 


(3.2) 


such  that  at  s  =  T,  "V"  is  given  by 

V(T,T,x(T),U(T)>  -  1/' (x(T)  ,T)  . . . . ......(3.3) 

The  following  assumption  on  the  regulator  problem  for  desired 
output  of  zero  are; 

(a)  The  plant  is  governed  by  the  linear  differential  equations 

dx(s)  “  F(s)  x(s)  +  G(s)  u(s)  . . . (3.4) 

ds 

(b)  The  loss  functions  are  quadratic  forms  in  the  state  and 
control  variables,  and  given  by  the  equations  (3.5) 


L(x(s) , u(s) , s)  =  1/2 


H(s)x(s)  1 1  2  +  ||  u( s) 1 1  2 

Q(s)  R(s) 


...... (3. 5) 


where  Q(s),  R(s)  are  symmetric  positive  definite  matries  of  class  C‘ 


Investigating  the  Euler  equations 

dx(s)  =  F(s)  x(s)  +  G(s)  R"^(s)  GT(s)A(s) 

ds 

d A(s)  -  HT(s)  Q(s)  H(s)  x(s)  -  FT(s)A  (s) 
ds 

and  making  the  change  of  variables  5  (s)  =  ~  A(s) 


(3.6) 


the  equations 


become 

dx(s)  »  F(s)  x(s)  —  G(s)R"l(s)G^(s)  f*  (s) 
ds 

-  df  (s)  ■  Ht(s)Q(s)H(s)x(s)  +  FT(s)  J  (s) 

ds 


(3.7) 
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And  the  control  law  becomes  from  equation  (2.13) 

u(s)  =  -  R"1(s)GT(s)  J(s)  ....................... ..(3.8) 

Because  it  is  desired  to  form  a  control  law  of  equation  (  )  a 

relationship  between  x(s)  and  5-(s)  must  be  established.  Recognizing 
that  the  Euler  equations  suggest  that  such  a  relationship  exists  we  let, 

f  (s)  -  P(ssT)  x(s)  . . .  (3.9) 

differentiating  this  relationship  with  respect  to  ”s"  we  get 
equation  (3.10). 

...,.,...,,.,.,..,..(3.10) 


dl(s) 

ds 


dP(s,T)  x(s)  +  P(ssT)  dx(s) 
ds  ds 


upon  substitution  for  d  J^(s)  and  dx(s)  for  equations  (3.7)  we  get 

ds  ds 

-HT(s)Q(s)H(s)  x(s)  —  F^(s)  J’(s)  =  dP(s »T)  x(s)  +  P(s,T)F(s)  x(s) 

ds 


“P(saT)G(s)R  (s)GT(s) %  (s) 


(3.11) 


which  becomes  the  well-known  Riccati  equation  stated  by  Kalman 

’  dP ( s ,, T)  -  P(s 3T)F(s)  +  FT(s)P(saT)  +  HT(s)Q(s)H(s) 
ds 

-  P(s9t)G(s)R“1(s)GT(s)P(sJT)  .................. ......(3.12) 

where  P(saT)  is  a  symmetric  matrix  to  avoid  trivia. 

This  is  the  same  result  as  that  obtained  by  Kalman  using  different 
techniques . 

We  now  generalize  the  above  problem  to  treat  the  case  where 
the  desired  output  is  given  by  z(s)  and  the  loss  function  is  given  by 


L(x(si)  r, u(s) )  «  1/2 


H(s)x(s)  -z(s)||2  4-  1 1  u(  s)  1 1  2 

"  Q(s)  ”  11  R(s)J 


.(3.13) 
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The  plant  is  assumed  to  be  given  by 

dx(s)  =  F(s)  x(s)  +  G(s)u(s)  +  K(s)  w(s) 

ds 

The  resulting  Euler  equations  are  given  by 

F(s)  -G(s)R-1(s)GT(s) \  /  x(s) 


(3. 14) 


(3.15) 


HT(s)Q(s)H(s)  FT(s)> 

/\  V  rp 

where  w(s)  =  K(s)  w(s)  and  w(s)  =  H  (s)Q(s)  z(s) 

If  we  can  put  this  in  the  form  of  the  equations  (3.7) 
F(s)*  -  R*(s)  \  /  x*(s)  \ 


Q*(s) 


FT*(s)y 


(3.16) 


\  }*<■>  / 

then  the  Riccati  equation  follows  immediately  as  above. 
The  above  form  can  be  obtained  under  the  transformations 


Let 

x*(s) 

=  x(s) 

I 

and  ^*(s)  = 

?  (s)  | 

Sc(s) 

F*(s) 

=  |F(s) 
0 

w(s)J 

R*(s)  = 

1 G(s)R 
0 

• 

(s)GT(s) 

Q*(s) 

“  |HT(s)Q(s)H(s) 

\  w(s) 

w(s)  | 

0  1 

G*(s)  - 

G(s) 

0 

The  result  Riccati  equation  is  given  by 

dP*(s,T)  -  P*(s,T)  F*(s)  +  F*T(s)P*(ssT)  -f  Q*(s)  - 

ds 

P*(s,T)R*(s)P*(s,T)  . .......(3.17) 


Kalman  has  suggested  that  the  performance  criterion  can  be 


expressed  in  the  nonnegative  definite  form. 
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P(S,T) 


(3, 18) 


46 , 


V(x,u,s) 


i  ||*(o|| 


if  U  (x(T))  =  1_  ||x||^  where  A  is  symmetric,  nonnegative  definite 

This  is  a  very  convenient  form,  because  (as  we  will  show  shortly) 

V(x,u,s)  is  then  a  Lyapunov  function, 

To  show  that  the  expression  (3,18)  obeys  the  necessary 

conditions,  recall  that  by  definition  ^(s)  “  P(s,T)  x(s).  We 

then  dif f erentiate  V (x, u, s)  with  respect  to  "s"  and  show  that  the 

right-hand  side  is  just  the  Hamiltonian  as  discussed  in  Section  I 

Part  II.  If  this  relationship  holds  then  the  expression  (3,18)  is 

the  performance  criterion. 

V(x,u,s)  -  1  xTP(s,T)  x  »  1  xT  (s) 

2  2 

Vg(x,u, s)  *»  1_  dxT  T  (s)  +  jL  xT  d  S  (s) 

2  ds  2  ds 

Substituting  the  value  for  and  dx  from  (3,7)  we  get 

ds  ds 

Vs(x,u,s)  »  =  ~L  ( x ,  u ,  s )  -(^^^(s)^ 

Furthermore,  if  a  plant  is  completely  controllable  then 
the  limit  as  T  — oo  exists.  We  denote  this  limit  by  P(t)  and 
denote  the  solution  of  the  Riccati  solution  (3,12)  by “["Rs ;A;T) . 
Therefore 

lim  TT  (s  ;A,T)  =  P(s)  . . . (3.19) 
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Furthermore,  if  this  limit  exists  the  performance  criterion 


becomes 


V(t;x,u,  oo) 

=  lim  V(t0;x,u,T) 
T  — •»  oO 


*  "S  <t0) 


(3.20) 


and  the  control  law  becomes 

u(s)  =  -R”^(s)G^(s)  P(s)x(s)  . . . (3,21) 

In  this  part  we  consider  the  stability  of  the  shrinking  time  to  go 
problem  as  stated  by  Kalman,  It  will  be  shown  that  if  a  plant  is 
uniformly  completely  controllable  and  uniformly  completely  observable, 
plus  the  additional  conditions  stated  in  the  following  theorem,  the 
plant  is  uniformly  asymptotically  stable.  If  we  weaken  the  conditions 
to  completely  controllable  and  completely  observable,  the  plant  is 
(non- uniformly)  asymptotically  stable. 

Stability  Theorem;  Consider  a  plant  with  control  law  (3.8) 
which  is  uniformly  completely  controllable  and  uniformly  completely 
observable.  In  addition  assume, 

(i)  L(x,u,s)  is  quadratic  and  of  form  (3.5) 

(ii)  o6I  >  Q(s)  >cx4I  >  0 

(iii)  C*7I  >  R(s)  >  cx  5i  >  0 

(iv)  ||$(s,T)||  <  (c*  s-T  |  )  for  all  s,T 
Then:  the  controlled  plant  is  uniformly  asymptotically  stable  and 
V(xj,s,  °o )  is  one  of  its  Lyapunov  functions. 
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Proof:  From  Lyapunov's  Theorem  (see  Appendix  B)  if  V(x,s,  oo) 
is  (a)  bounded  from  above  and  (b)  bounded  from  below  by  increasing 
functions  of  ||  x  ||  ,  (c)  the  derivative  V  of  V  along  optimal  motion 

of  the  plant  is  negative  definite  and  (d)  V — o£>  as  j|x||  — ►  »0 
then  V(x,s,oO)  is  a  Lyapunov  function.  Then  by  Theorem  (0-2) 
the  plant  is  uniformly  asymptotically  stable. 

Part  A  Recalling  the  proof  for  complete  controllability 

that  the  control  function  u*  (s)  could  be  properly  defined  by 

u1(s)  -  -GT(s)<|)T(t0,s)  W“L(t0T)  x0  . (3.22) 

if  we  substitute  this  into  the  equation  of  motion  (B-10) 

'tu1  (8ix0,t0)  -  ^)(a,t0)xo-  Cj)  G£7)GTcd)  <$T 

(Co.C')W-1  X0 . (3.23) 

Now  using  the  fact  that  (^(s,^)  "  ^ (s,t0)  Cj) 


the  equation  (3,23)  becomes 
■  $<s,t0)x0  -  (3^(s,t0)W(t0,s)W"1(t0,T)x0 

-  0(B,to)  [i  -  W(t0,a)W-l(t0,T)  ]  x0  . (3.24) 

because  T  >  s  we  assume  that  ||l  *"  W(tQ, s)W~*(t0,T)  J  |  T 

Define  z(s)  «  <^>  (T,t0)  [  I  -  W(t0,s)W_1(t0,T)  ]  xQ . (3,25) 

IUC»>I|  5  ||<J><T,t0)  ||-  ||  I  -W(t0.s)W-1(t0.T)||  •  ||  xc|| 

Therefore 

|1  z ( s)  1 1  <  CV8(CT)  ||x0|1  . (3.26) 

Because  the  region  defined  is  given  by  tQ,t0  +  0  (see  uniform 


completed  controllability)  where  (j  is  a  positive  constant  which 
transfers  xQ  to  zero  at  or  before  T  =  tQ  +  (j  .  Hence  o<  q((j  ) 
will  be  a  constant. 

NOTE:  (tQ  =  t  of  the  general  problem) 
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We  now  investigate  VCx.tQjO^).  Because  V(x,t0,00  )  -C 
V(x,t0T)  we  have  that 

T 

V(x,tc  oo)  <  f  L(x,u,s)ds  +  l  |1  x(T)  ||  2a  . . . (3.27) 

tQ  J  2 

where  A  is  a  symmetric  negative  matrix,  Using  this  along  with  ( i) » 
(ii)  and  (iii)  we  get 


T 

x,tQoO)  <  cx  e  ||  H(s)  (J)  (s,T)z(s)  ||  2  +  O^H^Cs 

to  r 

2  J  <x6  <*8  ||h<>)$(8.t)  || 2  •  ||*o|| 2  +  <*7  link. 

**0 

T 

-  f  «6  *  8  $T(s.T)HT(8)H(s)  <|)(S,T)  11x41  2 

to 

4-  0(7 (Xq  W-l(to>T)T(|)(tots)G(9)GT(s)(J)T(t0,s)  W’\ 

T 

1  «6  «8  MCX.t0)  ||x0||  2  +  of  7X0  W-l(Tt0,T)  x0 

^CX6<^8M(I,t0)||x0(|2  +  o<7  ||x0||2M-l(t0,T)  . (3.28) 

Thus  by  uniform  complete  controllability,  and  uniform  complete 
observability  we  have  that 

v(x,tOJo6)  <[o(6a8o<i*  id  )  +  o<7  O (Q(d  )  ]  ||x0  ||  2 

<  0<  9  ||  x0jj  2  . (3.29) 

which  proves  part  (a) 

Part  B  We  must  show  that  V(x,t0oO)  is  bounded  from  below  for 
increasing  ||  x  || 

From  V(x,tocx0)  «  ||x||2  _ 

P  <t0) 


V( 


ds 


ds 


ds 


<*10(to)  ||x 


Define  V(x,tQoo) 


(3.30) 


:  •:  -  '>■  v  1r  :  s  :■  ,  ■ 


!  © 


) .  !- H>*H  <y>  +  sij  ."!|(,  to > 

at  j  1  c :  V  r;  ,V  fT 

:  -  •:  J  ,  '  ... 

■  r.T.: 


(  -'e  ;  <r.  ■ 


if  V(x,tc,  oo)  is  bounded  from  below,  it  must  be  the  case  that 

cao<t0>  s  cxu  £° 


Assume  that  this  is  not  true  and  that  in  fact  there  is  some 
C(xss)  defined  by  ||x||2  C(x,s)  *=  f  ||  u  j|  2ds  ^  CX5  fllull^  ds 

t0J 


null2 

tQ J  R(s) 


=  V(x,  tQ  oo  ) 


. . (3,31) 

such  that  £(x,s)  can  be  made  as  small  as  desired  by  a  proper  choice 


of  x  t0. 


Define 


z(s) 


J** 


(tQJ s)G(s)u(s)  ds 


and  by  Schwary  inequality 
z(s)  ||  2 


-=(./»* 


(to#s)G(s)  ||  2  ds 


tj  UU<*>H 


ds 


by  uniform  complete  controllability 
||z(s)  ||  2S  W(t0,T)  |  ||  u(s)  ||  2  ds 


(3.32) 


(3.33) 


=  OC1(CT)||x0|| 


e(x0,t)  '  >IK 


(3.34) 


Utilizing  this  estimate,  we  find  with  the  addition  of  (i,)  (ii)  and 

(lii) 


V(x,t0,  o«) 


0<4 


H(s)  <^)  (s,tQ)  [  x0  +  (s)  ]  1 1  2  ds 


*0  II  -  I!  H(s) 


<j)  (s,tD> 


T  r  _ 

=  f  a^]ll 

■  1 1  Z(s)j  |  2J  ds 

?  0C4  ||$  (T,t0)x0||  2  M(T,t0)  -  0C12(  JMd.tjUx,, 

=- a  4  [a3'2(cT)0(*  (o') -alco")  «12(  > 


■  :  ;  ;V'.  ...  . 


•  -r1  '■ 


"  mx  X  whr  V  !  r>tu 


V  ,»)«  3T 


i  .  r  V,..  ■■  >■.  ,  ■  .  : 
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which  contradicts  the  assumption  that  hence  £  can  be  made 

arbitrarily  small. 

Part  c  Since  G  and  H  are  allowed  to  be  singular,  we  cannot  prove 
that  V  is  negative  definite.  However,  from  Part  B  we  have  that 


V<Qu(T,x,t0),  T,  oo)  ~  V(x9tQ  oo)  -  -[  OC  13  “  OC  ^  J  ||  x 
we  also  have  that 

V(Qu(T,xst0)  ,  T,  oo  )  -  V(x5t0,  ©o  )  2  -  CX5  ||  x  j  2 

setting  <Xi5  -  ot5a13  /  <a5  +cx14)  >  o'"' 

we  have 

O 

V(Qu(t0,  +  *  x,tQ),  t0  +  ,  oo)  -  V  (x,t0,  oo) 


.,.(3.36) 


(3.37) 


OC 


15 


x 


2  . .....(3.38) 


which  shows  that  V  is  decreasing  along  any  interval  of  time  of  length 
(j  unless  x0  »  0 


This  is  true  In  view  of  V 


x0 


Q.E.D, 


■ 
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SOLUTION  OF  THE  RICCATI  EQUATION 

The  Riccati  Equation 

-dP(s,T)  -  P(s,T)  F(s)  +  FT(s)P(s,T)  +  HT(s)Q(s) H(s) 
ds 

-  P(s,T)G(s)R_1(s)GT(s)P(s,T) 

can  be  solved  as  an  initial  value  problem  directly  by  the  solution  of 
the  2n  canonical  Euler  equations  as  illustrated  before. 


For  the  Euler  equations 


^(s) 

as 

F(s)  -G(s)R’1(s)GT(s)  \ 

x(s) 

ds 

-d|(s). 

as 

i  HT(s)  Q(s)H(s)  +  FT(s)  ' 

S-(B)  ' 

ds 


Let  the  transition  matrix  for  the  2n  Euler  equations  be 
r%.T>>  we  then  partition  this  matrix  into 


iy  11<S.T)  7 12<».T>' 

|721(s,T) 

It  is  known  that  S'(s)  and  x(s)  are  related  by  the  following 

equation  -  1^(s)  =  P(s,T)x(s) 

and  from  the  above  it  must  be  the  case  that 

dHs.T)  -  f  F(s)^G(s)R’1(s)GT(s) 
ds  ( 

^  Ht(s)Q(s)H(s)  FT(s)  ( 

so  the  solution 


x(s)  \ 

,z<«>  I 


X(T) 

Z(T) 


f  <s,T) 
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X(s)  =  ?^(s,T)  X(T)  +  712(s,T)  Z(T) 

Z(s)  =  72l(s’T>  X<t)  +  722^s^  z(t> 

because  Z(T)  -  P(T,T)  X(T) 

we  have 

X(s)  =  [r„u  »T)  +  ^2P(TsT)  ]  X(T) 

z(s)  =  [721(s,T)  +  722P(T,T)  ]  X(T) 

Hence  P(s,T)  -  Z(s)  X'1(s) 

Therefore 

P(s,T)  =  [72i(ssT)  +  7^22  (SsT)P(T9T)  ]  [7„  (S.T)  + 

7l2p(t>t) ]  -1 

For  proof  of  these  results  see  [  I  1-  ] 


. 

■ 


: 

( 
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EXAMPLE  PROBLEM  II  -  1 

To  illustrate  the  developed  theory  for  linear  systems  with 

linear  control  laws  consider  the  following  problem. 

The  plant  is  given  by  the  equations 

dx  =  ^llx  +  u 
ds 

y  =  x 

NOTE:  This  is  the  first  order  case  and  in  general  x  =  x; 

y  =  y,  and  u  =  u#;  however  for  convenience  we  drop  the 
subscript . 

WeThe  wish  to  optimize  this  plant  subject  to  the  performance 
criterion 


V 


mu2 


] 


ds 


Letting  *  q;  r^  »  r;  and  fn  =  f,  the  resulting 

Euler  equations  are 

dx  ■  fx  +  1  ) 

ds  r  A 

■  qx  -  f  A 

ds 

Rewriting  in  matrix  form 


Ik  1 

'  f 

i 

'  x  ' 

*1 

15 

r 

i 

q 

-f 

A 

From  the  matrix  (S)(S,t) 


sss 

1  cosh  a(s-t)  + 

f 

sinh  a(s- 1)  ,  1 

sinh 

a(s-t) 

a 

ra 

q  sinh  a(s- t) , 
a 

where  a2  = 

cosh  a(s-t)  -  £_ 

sinh 

a(s-t) 

f2 

a 

+  q_ 

r 


-  '  .  -  •  :s  ■  ,,  if  ::  +K  - 


. .  > 


»®t  'irJ: -  >  oi  .,Ve 
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The  control  law  for  this  problem  is  given  by 

u(t)  =  _1  •  A(t) 

r 

or  using  Kalman's  solution 

u(t)  =  -  l  P(t)  x(t) 

r 

Case  I  Natural  Boundary  Conditions 

Natural  boundary  conditions  are  given  by 

A(T)  53  P(T)  =  0 

Floating  Time  to  go  Solution 

Using  the  control  law  u(t)  =  1  A  (t),  it  is  necessary 

v  r 

to  find  A(t)  in  terms  of  x(t) .  From  the  transition  matrix  A(s) 
is  given  by 

A<  s)  =  £  sinh  a  (s-t)x(t)  +  cosh  a(s-t)A(0 

a 

~  _f  sinh  a  (s-t)  A(t) 

a 

Applying  the  boundary  condition  A('F)  "  0  gives  a  relation 

between  A<c>  and  x(t) 


A<o 


£  sinh  a(T-t) 
a 


£  sinh  a(T-t)  —  cosh  a(T-t) 
a 


-x(t) 


In  this  case  we  consider  fixed  values  of  t  and  successive 
values  of  T  to  observe  the  limit  of  C(T,t)  as  T  — ,  where  C(T,t) 
is  given  by 

ea(T-t)  -e-a(T-t) 


e»d:0(f 


(£  +  a) 


;  -i  :;0;  \1  f  ±o 


.  *  '  ; 
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In  the  limit  as  T  — m  ,  C(T,t)  tends  to  3 _ 

f-a 


Shrinking  Time  to  go  Solution 

Because  we  have  subjected  this  problem  to  natural  boundary 
conditions,  Kalman's  solution  may  be  used.  From  which  it  follows  that 

%i.(t,T)  +  722<t:>T>p(T>T>- 


P(t,T) 


J  n(t,T)  +  J12(t,T)P(T,T) 

Because  A<T>  =  0  and  x(T)  ^  0  it  must  be  the  case  that 

P(T,T)  «  0. 

In  the  solution  of  the  shrinking  time  to  go  problem,  T  is 
fixed  and  successive  values  of  t  are  considered  for  t  £.  T. 


P(t,T) 


j(ea(t-T)  ,e-a(t-T)| 


(a  4.  f)  aa(t-T)  H-(a  -  f)e"a^t"T) 

Taking  the  limit  P(t,T)  tends  to  g _ 

f-a 


For  the  steady  state  case,  an  easier  method  is  available  for 
finding  the  limit  of  the  shrinking  time  to  go  problem.  This 
solution  is  obtained  by  setting  the  right-hand  side  of  the  Riccati 
equation  equal  to  zero  and  solving  the  resulting  quadratic  equation 
for  the  nonnegative  definite  solution.  This  is  given  by, 

P  a  SL_ 
a-f 
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Case  II  Boundary  Conditions  x(T) 


0 


Floating  Time  to  go  Solution 

From  the  transition  matrix  x(s) 

cosh  a(s-t)  +  f_  sinh  a(s-t)  x(t) 
a  J 


is  given  by  x(s)  « 


+  sinh  a(s-t)  /\(t) 

ra 

Subjecting  this  equation  to  the  boundary  condition  x(T)  =  0 

gives  the  desired  relationship  between  ,\(t)  anc*  x(t). 

This  relationship  is  given  by 

A(t)  =  [— ra  cosh  a(T-t)  +  f  sinh  a(T-t)]  x(t)/ 

L  “  J  /sinh  a(T-t) 

In  this  case,  as  the  time  to  go  becomes  infinitesimal,  there  remains 
some  error,  hence  the  control  effort  asked  for  becomes  unbounded  in 
an  attempt  to  satisfy  the  end  boundary  conditions.  This  can  be 
seen  by  evaluating  C(T,t),  at  t  =  T  where  the  sinh  a(T-t)  becomes 
zero.  Hence,  C(T)  becomes  infinite.  In  this  case,  the  control  law 
is  said  to  have  finite  escape  time. 

Similarly  we  could  solve  this  problem  by  using  the  shrinking 
time  to  go  method.  Note  however,  that  because  P(T)  is  not  defined. 
Kalman's  solution  is  not  applicable. 


SECTION  II 


PART  IV  CONSTRAINTS  IN  VARIATIONAL  PROBLEMS  AND  SOME  NONLINEAR 
CONTROLLERS 


In  the  procedures  developed  thus  far  no  account  has  been 


made  for  possible  limitations  on  the  allowable  range  of  the  state  of 
control  variables,,  their  derivatives  or  integrals.  The  treatment  of 
a  large  variety  of  constraints  can  be  done  by  the  use  of  Lagrange 
multipliers,  this  method  has  already  been  employed  to  convert  a 
performance  criterion  constrainted  by  the  plant  into  an  unconstrainted 
performance  criterion.  Constraints  on  the  plant  and  the  control 
functions  which  are  most  easily  solved  by  the  use  of  Lagrange 
multipliers  are  generally  of  the  type 


(4.1) 


q(t;x(t) ,u(t))  =  0 


where  "q"  is  an  r-dimensional  vector  function. 

The  performance  criterion  can  then  be  modified  by  the  addition 
of  the  term 


(4.2) 


q(s ;x(s) , u(s) )  ds 


t  J 

(where  y0( s)  is  the  Lagrange  multiplier). 

and  require  that  the  performance  criterion  be  stationary  with  respect 
to  p ( t)  as  well  as  the  other  variables.  The  modified  performance 
criterion  will  be  stationary  only  if  equation  (4.2)  is  satisfied; 
then  the  value  of  the  modified  criterion  will  be  equal  to  the  original 
"V".  This  procedure  will  introduce  into  the  Euler  Lagrange  equation 


an  extra  set  of  algebraic  equations,  equation  (4.1)  in  the  variable 


' 
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Another  possible  form  of  a  constraint  is  the  inequality 


integral 

T 


J  qi(s;x(s)  , u(s))ds  <z  [7 


o  ©  ©  «  o  o  o 


..(4.3) 


where  (i  -  1,2,3,  ........... ,n) 

be  satisfied.  One  again  can  introduce  a  set  of  constant  Lagrange 
multipliers  "fjL  M  and  add  to  the  performance  criterion  a  term 
T, 


■q(s;x(s) ,u(s))  ds  .............................. (4. 4) 


and  carry  out  the  optimization  for  a  range  of  values  of  the  multipliers 
" expressing  the  optimum  "V"  as  a  function  of  " fj^' •  One  then 
performs  what  is  but  static  optimization  with  respect  to  subject 

to  the  constraints  of  equation  (4.3). 

Another  inequality  constraint  which  is  the  most  important 
type,  from  a  physical  point  of  view,  is  the  form 

uj -4  uj/t)  <£  u^  (i  »  1,2,3. ,n)  ........  (4. 5) 

xi  <  xi(t>  ^  xi 

where  the  superscripts  u  and  1  designate  upper  and  lower  limits  or 
bounds  respectively.  These  types  of  constraints,  especially  the 
condition  on  u^(t)  appear  in  physical  situations.  Some  examples  of 
problems  of  this  type  that  have  been  solved  to  date  are  the  minimum 
time,  fuel  and  energy  problems.  There  are  several  ways  of  incorporating 
these  inequalities.  The  most  simple,  although  not  the  most  accurate,  is 


to  substract  from  "V"  a  term  of  the  form 

Tr  Tr 

I  Pu  (ui(s))ds  and  /  Px  (xi(s))ds 

t  J  1  t  J 


(4.6) 


' 
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where  P  and  P„  are  penalty  functions  for  the  control  and  state 
functions  respectively.  These  penalty  functions  are  such  that  they 
provide  a  large  penalty  function  to  "V"  if  their  arguments  are  about 
to  violate  the  bounds  expressed  by  equation  (4.5)  but  little  if  their 
arguments  are  within  bounds.  The  most  convenient  function  of  this 
kind,  that  applies  to  a  large  variety  of  practical  problems,  has  been 
found  to  be 


puiK> 


ui 

or 

where 

u 


V.2^  +  (k2  -  kj)(I  +  2u) 

kjU^ 

k2U  +  (k2  -  kj)(I  -2u) 

pUiCui)  (u>r 

Ui  -  I/2(uJ  -  uj-) 

— ■ 


u  >  -I 

U  |  <  I 

u  >  I 


(4.7) 


i  ui 

and  k2^kj  and  r  is  as  high  an  even  power  as  is  necessary  to  achieve 
a  sharp  enough  constraint, 

As  was  pointed  out  this  is  just  an  approximate  method, 
However,  it  has  the  advantage  that  it  is  easy  to  apply  in  complicated 
problems  and  can  be  used  to  constrain  both  the  state  and  the  control 
variables , 


Another  way  of  handling  inequality  constraints  of  the  nature 
described  above  which  leads  to  an  exact  solution  is  to  use.  a  change 
of  variables,  If  y(t)  is  some  variable  which  is  to  be  constrainted 
between  two  limits  y*  and  yu,  one  introduces  a  new  variable  z(t) 
such  that 


y(t) 


q(t;z(t)) 


(4.8) 
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and  q(t;z(t))  remains  in  the  bounded  region  specified  by  the  constraints. 
Substituting  q(t;z(t))  for  y(t)  one  then  performs  the  optimization  with 
to  z(t),  thus  for  any  z(t)  that  may  result,  y(t)  will  remain  within 
the  constrainted  limits. 

To  illustrate  the  use  of  the  method  presented  for  the  solution 
of  problems  subject  to  constraints,  let  us  consider  the  problems  of 
minimum  time,  fuel  and  energy.  We  shall  solve  these  problems  for 
necessary  conditions  by  the  use  of  Euler  equations. 

The  Minimum  Time  Problem 

The  minimum  time  problem,  or  the  "bang-bang”  problem  has  been 
examined  in  detail  during  the  past  ten  years  and  the  solution  of  an 
idealized  relay  is  well-known.  The  reason  that  this  problem  is 
presented  here  is  to  compare  the  developed  theory  with  a  well 
established  result. 

The  plant  described  by  equations  (4.9-10) 

dx(s)  *=  Fx(s)  +  Gu(s)  . . . .....(4.9) 

ds 

y(s)  »  Hx(s)  . (4,10) 

and  the  control  u(s),  restricted  in  magnitude  by  equation  (4,11), 

uj(s)  5L  Cj  . . . (4.11) 

where  J  -  1,2,  .  p 

for  simplicity  we  will  assume  that  all  cj  are  eqyral  to  "1". 

It  is  assumed  that  the  system  is  controllable  (see  Section  I, 
Part  2).  The  initial  and  final  states  to  the  plant  are  given  by  xc 
and  Xf  respectively.  With  no  loss  of  generality  assume  t  =  0. 
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For  the  minimum  time  problem  the  loss  function  will  be  given 
by  L(s;x(s) , u(s))  =  I  since 

T  i 


7 

0  J 


L(s(s) ,u(s))ds  =  T 


To  account  for  the  constraints  imposed  on  the  control  function 

m 

u(s)  we  add  the  penalty  function  of  the  form  ]T  uj(s)  r  to  the 

j=I 

loss  function  where  each  component  uj(s)  of  u(s)  is  penalized  by  the 

function  uj(s)  .  r  is  given  to  be  some  even  power  as  high  as  necessary 

to  achieve  a  sharp  enough  constraint.  The  new  loss  function  will  be 
m 


given  by 


i  +  E  f  u,(s)  1 

i=I  L  J  J 


J 

The  resulting  Euler  equation  for  this  system  are 
Fx(s)  +  Gu(s) 


dx(8) 

ds 


IlsI 


-  FTA(s) 


(4.12) 


ds 

gU(s)  =  ru(s)r  “  J 

Because  u(s)  must  be  subjected  to  the  constraint  |uj(s)j^  1  we  may 
rewrite  the  last  equation  as 


u(s) 


sgn 


a  (s) 


I  GT  A(s) 


and  in  the  limit  as  r— v-°othe  term 


r  -  i 
I 


becomes  equal  to  I 


and  a  necessary  condition  for  the  minimum  time  problem  is  that 
U  (  S  )  D  S  gn  ^  q  (s)  ] 

This  situation  is  depicted  in  Figure  (II* 4. 1 ^A) 

NOTE;:  (  q  (s)  is  defined  to  be  GT  (s)) 


■ 


J  ,! 


The  Minimum  Fuel  Problem 


There  are  many  problems  where  the  control  variables  u(s) 
are  directly  proportional  to  the  rate  of  flow  of  mass.  For  example 
the  control  signal  may  be  the  flow  from  a  gas  jet  for  the  attitude 
control  of  a  space  vehicle  or  the  exhaust  from  a  missile.  Almost 
invariably  in  these  problems,  the  total  fuel  available  is  limited; 
therefore,  it  is  desirable  to  accomplish  each  control  correction 
with  a  minimum  amount  of  fuel. 

In  this  case,  the  loss  function  will  be  given  by 


m 


u,(s) 

0 


To  solve  this  problem,  once 


Lx( s) ,  u(s)  =  L 

j-1 

again  we  add  for  each  input  a  constrainted  function  u^(s)  =  sin 

m  J 

resulting  in  new  loss  function  namely  X.  (  sin  &  A  from  which  the 

j-1 

Euler  equations  are 


dx(s)  =  Fx(s)  +  Gu(s) 
ds 

d  A  (s)  =  -  FTA(s) 

ds 


,,.(4.14) 


where 


GT  A(s)  -  L 
ujCs)  |  <  1 


We  now  define  the  vector  "q"  to  be  equal  to  GT  \  (s),  the 
last  equation  of  (4.14)  becomes 

cos  S  q(s)  =  cos  ^  . (4.15) 

for  |  uj(s)  1  —  1 


Case  1: 

then 


If 


uj<8) 


qj(s) 

*  sgn 


>  1 


qj(s) 
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Case  II:  if  |qj(s)|  <  I, 

then  uj(s)  =  0 

Thus,  for  minimum  fuel  operations,  the  control  law  is  given  by 
u j ( s)  =  0  if  |  q(s)  |  <  1 

Uj(s)  =  sgn  J  q  j  C  S>  ]  |  q..(s)  |  ]>  1  . (4.16) 

From  which  we  can  conclude  that  the  values  of  optimal  control  functions 
are  £  1,  0  or  -l"^  . 

This  situation  is  depicted  in  Figure  (II  4.1,B)  below. 
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The  Minimum  Energy  Problem 

There  exists  a  class  of  problem  s  for  which  the  square  of 
the  control  signal  is  proportional  to  power,  and  the  time  integral 
of  the  square  of  the  control  signal  is  a  measure  of  the  energy 
dissipated.  As  an  example  suppose  that  the  control  u(s)  is  an 
electric  signal  deriving  its  energy  from  a  battery,  and  it  is 
desired  to  accomplish  a  given  control  action  using  a  minimum  amount 
of  energy  provided  by  the  battery. 

2 

The  loss  function  in  this  case  will  be  a  function  of  u(s) 
and  given  by 


(4.17) 


L(x(s)  ,  u(s)  )  =  1/2  £  [  u  .  (  s )  1  2  =  I  <'u(s),u(s)>/> 

j-1  J  J  2 

subject  to  the  constraint  |  u j ( s )  j  K  1 
To  account  for  the  bounded  condition  imposed  on  u(s)  let  u j ( s )  =  sinfrj 
this  condition  remains  in  the  bounded  condition  imposed. 

The  control  law  in  this  case  will  be  given  by 
cos  ■&  q(s)  =  cos  ■&  sin-© 

From  this  equation  we  see  that  several  conditions  exist 
Case  I:  if  |  q j ( s)  j  >  I  then  it  must  be  the  case  that  cos  =  0 
hence  sin  Sj  *»  I,  hence  uj(s)  a  sgn  |  qj(s)  j 

Case  2:  if  J  q j ( s)  j  <  I  then  the  equation  exists  for  all  values 
j  ■  1 ,  ......  m  ,  hence  uj(s)  »  qj(s) 


In  conclusion  we  see  that  the  optimal  control  law  for 
minimum  energy  is  given  by 

q j ( 8)  if  |  qj(s) |  <  I 


Uj(s) 

uj(s) 


ign  q-j(s)  if  |  qj(s)  |  J>  I 


(4.18) 
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EXAMPLE  PROBLEM  II  -  4.1  A  Complete  Solution 

The  Minimum  Fuel  Control  System  for  a  second  order  system. 
We  have,  using  Calculus  of  Variations s  developed  a  necessary 
condition  for  the  minimum  fuel  problem.  The  conditions  are, 
uj(s)  -  0  if  |  q j ( s)  |  <  1 

u(s)  =  +  sgn  £  qj(s)}  if  |  qj(s)  |  >  1 

and  A  (s)  satisfies  the  equation 
A(s)  =  -  FT  A  (s) 

and  the  solution  for  the  above  system 

A(s)  'f|e~^(s)  where  A(0)  =  V 

We  now  consider  a  specific  second  order  problem  and  solve  the 
complete  problem  by  use  of  phase  plane  analysis. 

Equation  of  motion  is  given  by 
d^x 

-  u(s) 


Putting  this  equation  into  Jordan  canonical  form, 


0  1 
0  0 


x  1  (  s  ) 
X2(s) 


It  is  desired  to  force  the  plant  from  an  arbitrary  state 
Xj[(0)  ■  JA  (i  *  X j, 2  )  to  the  orlgin(0,0)  using  a  minimum 

amount  of  fuel 


subject  to  the  constraint 


u(s)  |  <C  1 


. 
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From  necessary  conditions  we  see  that  u(t)  must  obey  the  equations 
u(s)  *  0  if  j  A7(s)  |  <  1 

u(s)  *  -sgn  [A  2(s)  j  if  |  A  2(s)  |  A  1 
Hence  for  minimum  fuel  control  u(t)  =  0  or  u(t)  E  A  =  +  1 

To  determine  the  sign  of  u(t),  it  is  necessary  to  obtain 
information  about  the  adjoint  system.  The  solution  for  the  adjoint 
system, 


A]_(s)  \ 

[  A2(s>  / 

is  \(s)  =  7X 

A2(s)  =7  2 


f 0  M 

'Ax(s) 1 

(-1  0  j 

l A  2(s)  , 

Vl  s 


A  (0)  =  7i(i  =  1,2) 


As  was  pointed  out  in  the  previous  examples,  the  relation 
which  exists  between  x^  and  x2  and  "7  i  and  ”7  2  is  possibly  nonlinear 
and  unknown.  For  this  reason  we  consider  all  possible  forms  of  the 
Al(s)  and  A  2^s)  to  determine  the  possible  control  sequences. 

There  are  six  possible  solutions  of  A  2(s)  and  are 
illustrated  in  Figure  (11-4. 2). 

A  control  sequence  A  »  0j  —  A  means  that  first 
u(s)  =  A  is  applied,  then  u(s)  =  0,  then  u(s)  =  —  A  is 

applied. 


From  the  above  figures,  the  allowable  sequences  are 

[o],  (A]  .  [o.  A]  .  [A,o]  ,  (A,o, -A} 

To  reach  the  origin  with  a  minimum  amount  of  fuel,  one  of 


the  above  sequences  must  be  used. 
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for  u(s)  =  0 


(4.21) 


u(t)  =  A  =  +  i 


We  mast  now  investigate  the  state  solutions.  The  solutions 
of  the  state  equations  are 

xl<s>  =  Jl+  f2t 

x2(s)  -  %  2 

and  x-^(s)  *  ^  +  J^t  +  1  /]  t2 

x2(s)  -  2  +  A  t 

Let  us  first  consider  the  plot  in  the  phase  plane  which 
forces  (  ^  %  2^  to  (0»Q)  using  the  control  u(s)  =  0,  this  set 

contains  only  one  point,  the  origin  itself. 

Consider  the  set  of  all  states  in  the  plane  which  may  be 
forced  to  (0,  0)  using  u(s)  °  +  1  or  -1,  Solving  for  x^(t)  in  terms 
of  X2(t)  we  get 


xi  - 

and  xj^(T) 

hence  for  u(s) 
u(a) 


£i  +  |  A 


{All 


I  A 


x; 


x  (s)  ** 

,  2 

1  X, 

r°  <  x 

2 

Vx2  —  0 

xCs)"  _1 

2 

X2 

Cxx<  0 

2 

Lo  <  x 
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Denote  these  sets  as  ^  +  and  'tf  _  respectively.  We  now  break  the 
remaining  plane  into  four  regions  as  shown  below. 


These  regions  are  defined  as  follows, 

Region  X  is  the  set  of  points  such  that  >  0  ;  xj  >V  ~  -1 

2  2 

Region  XX  is  the  set  of  points  such  that  X2  >  0,  xj  <OL. 

Reglon  XII  is  the  set  of  points  such  that  X2  <  0,  xj  <  Y + 

Region  IV  Is  the  set  of  points  such  that  Xj  ">■  Y+,  X2  <  0 
We  now  consider  the  necessary  sequence  to  drive  the  plant  from  some 
initial  state  <  X 1.  S  2>  to  zero  for  minimum  fuel, 

Let  (  £  i,  2)  be  a  point  on  the  curve  +  .  The  only 
control  sequence  which  will  force  (  f  2)  to  the  origin  is 

jf+I  j  any  other  sequence  cannot  force  ( £  ± J  2)  to  the  origin. 
Similarly,  the  control  £-l_f  will  force  (  £  2)  on  to  the  origin. 
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Let  us  now  consider  Regions  IV.  There  are  several  control 


sequences  which  could  drive  (  J  2)  to  the  oic igio s  it  is 

necessary  to  determine  the  one  which  does  so  with  minimum  fuel. 
The  sequences  which  work  are  ^J3S  +  I J  „  £  ~ISQS  1^- 

For  the  sequence  ^0S  -I-  I  y  when  u(t)  =  0  is  applied 
x2  “  &  2  anc*  xj  ~  S  l  +  X  2't  where  J  2  is  negative 9  hence 
xj  will  decrease  as  "t"  increases  and  X2  will  remain  constant  to 
(  ^  i»  T  2) •  This  situation  is  illustrated  in  the  figure  below 


Then  the  application  of  +1  will  drive  the  plant  to  the  origin 

In  this  ease  we  compute  the  fuel  required  to  be  F  *=  (  jf  2  j 

If  the  control  sequence  ^-1,0,,  I  j  is  used9  application  of 

U  ,,// 

u  ■>  "I  will  force  the  state  C  S-  lo  ^  2)  a  8tate  (j  in  ^  2) 

I  "  0 

2  .  At  (  J  ^  2  )  we  apply  u  »  0  which  drives 

<  fu  fi  )  «  (  f'v  Ta>  "*  +  • 

///  ,  //  ''I  1/ 

At  <  S  Ip  J  2  )  we  aPp!y  u  "  I  to  drive  (  §  1#  j'2  )  to  the  origin. 

„///  11 

The  fuel  in  this  case  to  reach  the  origin  from  (  %  \9  J*  2  )  *8 

J 

the  minimum  sequence  required, 


>  If 


Hence  the  sequence  ^  0,  -I-  I  J-  is 


where 


1 2I  > 


. 


, :  !:;■  ■  ,  ,  ,  : 


I  v  t. 
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■  w 


We  can  do  the  same  thing  for  the  Region  II  to  get  the 


minimum  sequence  £0,  -Ij  , 

Region  I  and  III,  however,  are  more  interesting  and  it 
will  be  shown  that  one  must  first  drive  (  J*  2)  into  the  adjacent 
region  and  then  to  the  origin. 

Consider  the  Region  I  where  the  allowable  sequences  are 
£-1,0,  i]  in  this  case  u  =  -I  is  applied  and  drives  the  point 
(  ^  1^2^  into  Region  IV  where  the  sequence  £0,  1^  will  then 
drive  it  to  the  origin.  Similarly  the  sequence  ^IjOj-I^J-  is  the 
minimum  sequence  for  Region  IV. 

The  allowable  sequences  for  the  various  regions  are  listed 


below 

Optimal  Control  Law  for  Minimum  Fuel  Operation 


State  Point 

On 

On 

Region  IV 

Region  II 

Region  I 


Region  III 


Optimal  Control  Policy 
Apply  u(t)  "  I  to  reach  the  origin 
Apply  u(t)  »  -I  to  reach  the  origin 
1/  Apply  u(t)  B  0  to  reach  Xj-then 
2/  Apply  u(t)  "  I  to  reach  the  origin 
1/  Apply  u(t)  0  0  to  reach  V.  then 
2/Apply  u(t)  »-I  to  reach  the  origin 
1/  Apply  u(t)  = -I  to  Just  enter  Region  IV 
2/  Apply  u(t)  ■  0  to  reach  ‘8^  then 
3/  Apply  u(t)  “  I  to  reach  the  origin 
1/  Apply  u(t)  »  I  to  just  enter  Region  II 
2/  Apply  u(t)  ”  0  to  reach  X.  then 
3/  Apply  u(t)  “  -I  to  reach  the  origin 


. 
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Let  us  now  extend  the  Minimum-Fuel  second  order  system  to 
a  third  order  system  and  develop  a  partial  solution  to  illustrate 
some  of  the  computational  difficulties  that  arise  when  we  extend  to 
higher  order  systems.  To  the  authors  knowledge  this  problem  has  not 
as  yet  been  solved. 

The  canonical  system  for  this  problem  is  given  by 


and  this 


0  0  o\ 


u(t)  \ 


/ 


(4.23) 


system  is  subjected  to  the  initial  conditions  (  T  2*  T  3) 

The  adjoint  system  is  given  by, 


I  y  1  \  /  °  -1  °  \  /  1  \ 

X2  j  -  [  0  0  -I  2 

\  A3  /  \  0  0  0  /  \  3  / 


(4.24) 


and  this  system  is  subjected  to  a  set  of  unknown  initial  conditions 


<  ti,  Tz  f3> 


From  the  previous  example  we  see  that  the  necessary  conditions 

for  optimal  control  are  given  by  the  equations 

u(t)  ■  0  if  I  A  (t)  |  I 

*  . (4.25) 

u(t)  -  +sgn  )  X  (t)}  if  I  X  (t)  I  >  I 

The  solution  of  the  adjoint  system  will  provide  information 

on  the  admissible  sequences  for  an  optimal  control. 

Solutions  of  the  adjoint  system  are  given  by 

Xi  =  7  3  fc2/2  ^2t  + 

X2  “  "  ^3  t  +72 
A  3  =  73 


(4.26) 
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By  investigating  the  possible  solutions  for  °l  ^  (t)  we 
find  that  the  allowable  sequences  ares 


[0] 

$> 

<1 

^ — ) 

<3 

O 

oO 

<1 

s0  J  , 

[A  ,o, 

-  A],  £  A  ,o,  A] 

1 0, 

A 

»°] 

.  (o.  A  ,o,-  A3  . 

i  A  , 

o,-  A  so 

CTl 

O 

<3 

o 

<3 

x/ 

We 

now  consider  the  solution 

of  the 

:  system 

If  u 

i(t) 

0 

then 

X1 

-  Ii 

x2 

"  JV  +  /  2 

x3 

■  J>2/2  +  J>2t  + 

If  u 

( t) 

as 

A  -  + 1 

X1 

■  At  +x 

x2 

»  A  t2/2  +  jT  ]_t  + 

S2 

x3 

*  A  b^/6  +  $ |t2/2  + 

Sit 

+  J  3 

Or  in  this  case  it  is  more  convenient  to  solve  these  problems  non- 
parametrlcally. 


Putting  the  equations  in  the  form 

dx  :  d*2  '  &K3  »  u(t)  :  x  : 

*  1 

If  u(t)  »  0 
then  xi  ■  f  , 

J  . .  (4.29) 

x2/2  “  2/2  “  I(x3  “  cT  3^ 

If  u(t)  =  I 

xl  -  J  i  “  2  <-*2-  2)  . . (4.30) 

6  A2<*3  -  S i>  -  xi  ~  fi  +  22  A  “  £l><xi  -  3,) 


then 
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As  before  we  now  investigate  the  curve  where  u(t)  =  0  will 
drive  any  initial  point  (  £  i>  ^2*  S  3^  to  t^e  term^na^  point(0,0,0) 
Therefore  it  must  be  the  case  from  equation  (4.29)  that 
X1  (T)  33  0  hence  ^  ^  =  0 

however,  because  X£  and  x^  are  also  zero  at  "T  "  so  are  ^  ^  and  ^  y 


Hence,  the  path  in  the  phase  plane  which  forces  (  ^  , *  JT  2  »  T  3)  to 
the  terminal  point  (0,0,0)  using  the  control  u(t)  =  0  contains  only  one 
point  ,  the  origin. 

Let  us  now  investigate  the  curve  where  u(t)  =  f  I  will 
drive  the  initial  point  to  the  terminal  point. 

In  this  case  the  curve  is  defined  by  the  intersection  of 
the  two  surfaces  below  in  the  region  x^  >  0  and  X3  >  0 


1/2  x" 


1/6  xJ  =  X., 

1  3 


X+  . (4.31) 


This  situation  is  illustrated  in  Fig.  (II-4.3) 

Similarly  we  can  do  the  same  for  u(t)  ■  -I 
Let  us  now  consider  one  admissible  control  to  illustrate 
the  method  which  must  be  applied  to  determine  the  desired  control 
law.  Consider  the  sequence  £  0,  • 

Thus  we  must  find  conditions  under  which  the  curves  for  u  (t)  =  0 
intersect  the  curve  $ y. 

Curves  for  u(t)  «  0  are  given  by  the  equations 
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and  the  surface 


The  intersection  of  the  planes  x-j  = 
x22  =  2  <f  lx3  “  2  f  l  f  3  +  J  2  form  the  space  curve  on  which 

u(t)  =  0  will  force  the  plant.  We  now  find  what  values  of 
(  f  1»  t  2>  S  3^  will  cause  this  curve  to  intersect 

e  2 

For  this  to  occur  it  must  be  the  case  that  1/2  £  i  =  X£ 
and  1/6  ^  *  x^.  Hence  the  following  equation  must  be  satisfied 

Si  -  u  f iX3  +  12  J22  -  0 . <4-32> 

Consider  the  specific  example  where  the  initial  point  is  given 
by  (1,  3,  4  5) 


Upon  substitution  into  equation  (4,32)  we  see  that  this  point 
satisfies  the  necessary  condition.  This  curve  will  intersect  the 
curve  at  the  point  (I, 1/2, 1/6)  as  shown  on  Fig. (11-4,3),  This  example 
is  used  to  illustrate  the  difficulty  in  obtaining  a  complete  general 
solution,  The  complete  solution  for  this  problem  is  as  yet  not  solved. 


FIG  II  (4-3) 
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SECTION  III 

Part  I  LINEAR  FILTER  AND  PREDICTION  THEORY  AND  ITS  RELATIONSHIP 

WITH  THE  LINEAR  REGULATOR  PROBLEM 

Classical  results  based  on  Wiener's  theory  ^15,  19 j  in  the  field 
of  filtering  and  prediction  have  been  studied  in  detail  for  stationary 
cases  by  the  use  of  transform  techniques.  For  stationary  problems  the 
synthesis  of  a  filter  is  done  by  the  method  of  "Spectorical  Factorization". 
These  results  where  extended  to  the  time  domain  for  the  solution  of 
nonstationary  problems  by  Steeg  | 2 lj  and  Shinbrot  ^20 j  using  impulse 
response  methods.  The  difficulty  in  the  nonstationary  case,  once  the 
impulse  response  has  been  determined,  is  to  synthesize  a  filter  from  this 
information,  this  of  course  is  not  easy  because  the  synthesis  of  an 
integral  equation  to  a  system  of  differential  equations  is  not  unique. 

This  problem  is  sometimes  called  the  identification  of  the  system  from 
its  impulse  -  response  matrix.  Appendix  B  . 

In  this  section  the  classical  Wiener  -  Hopf  Equations  will  be 
synthesized  by  a  system  of  differential  equations,  this  is  carried  out 
in  Appendix  (C) . 

Optimal  Filter  Problem 

The  optimal  filter  problem  will  be  formulated  under  the  following 
assumptions . 

( A^ )  Assume  that  the  input  is  composed  of  two  random  signals,  one 
being  the  desired  and  the  other  being  the  undesired  tending  to  obscure 
the  desired  input.  The  input  is  given  by  the  equation 

z(t)  =  v(t)  +  H(t)x(t) 


(1.1) 


.  . 


j  :  •  | 

■Jl.  ,  r  i.  y-'  >s:  ?  ic,:  Un  ; 

-.13 .  iji / ;  3 .  _  x<a  |1.|  .  _£  te  f  3  9  0 

‘  MS  fJ  «  . 

-  n  3;  '  ".XJ  Sj'i-1  :  tf'Jo  b .9  •  .  .  "  .  nx:  ' 

(  {  ■  :t ) ,? 


80„ 


where  v(t)  is  the  undesired  signal  (process)  and  x(t)  is  the  desired 

signal  (message). 

(A2)  Assume  that  the  undesired  process  v(t)  is  white  noise  and 
independent  from  x(t) 

(A3)  Assume  that  the  message  x(t)  can  be  generated  by  passing 

white  noise  through  a  linear  system  given  by 

dx  =  F(t)x(t)  +  G(t)u(t)  . . . .  . . . ..,(1.2) 

dt 


where  u(t)  is  an  independent  random  process  (white  noise), 

If  the  above  assumptions  hold  and  in  addition  we  assume  that 
u(t)  and  v(t)  have  identically  zero  means,  the  covariances  will  be  given 

by 


■  Q(t)'  5  (t  -  f) 

.  j  v(t)  ,v(T  )  ]  “  R(t)*  Jj(t  -Y)  ..............  (1.3) 

»  0 

(A^)  The  measurement  of  z(Y)  starts  at  some  fixed  instant  "t0" 


cov, 

U(t),u  (f  )  j 

cov, 

|  v(t)  ,v(T  )  ] 

cov. 

v(t),u(T)  ] 

is 


of  time  (which  may  be  -  <x>)  ,  at  which  time  the  cov.  x(tQ)  ,x(tQ) 
known.  In  the  special  case  when  the  dynamic  system  has  reached  steady 
state  under  the  action  of  u(t),  then  x(t)  is  a  random  function  defined 

by 

x(t)  - 


_  00 


J  $  (tsr)G(rwr >dT  . . ......(1.5) 


This  formula  is  valid  only  if  the  limit  as  tG — —  —00  exists  and  is 
uniformly  asymptotically  stable. 


Under  the  above  assumptions  we  can  now  solve  the  "Optimal  Estimation 
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Problem" . 


Define  the  optimal  estimate  of  x(t^)  to  be^xCt^  J  t)  ,  where 
t-^  »  t,  t^  <  tjOr  t|  ]>  t, 

If  t^  t  we  have  a  prediction  problem. 

If  t^  =  t  we  have  a  filter  problem. 

If  t^  <  t  we  have  a  data  smoothing  problem.  (This  problem  is 
not  treated  in  this  paper). 

The  "Optimal  Estimation  Problem",  which  is  a  generalization  of 
well  known  "Mean-Square  Error"  used  in  classical  stationary  problems, 
is  stated  as  follows. 

Given  known  values  of  z(7”)  in  the  time  -  interval  t0<T  <  t, 
find  an  estimate  x^t^  |  t)  of  x(t^)  of  the  form 


x(tx  |  t)  »  J  T)z(f)dT 


.(1,6) 


(where  A  is  an  nxp  matrix  whose  elements  are  continuously  dif f erentiabl 
in  both  arquments  )  with  the  property  that  the  expected  squared  error 
in  estimating  any  linear  function  of  the  message  is  minimized: 


£  [x*,  x(tx)  —  x(tL  |  t) 


minimum  for  all  x  ,..,(1,7) 


where  x  is  the  costate  of  x  see  Appendix  A 

A  special  case  of  the  variance  estimator  of  the  loss  function 
is  the  more  well  known 


l|  *Cci) 


x^t^  |  t) 


2  . . 

The  optimal  estimation  problem  is  depicated  in  Fig.  (3-11). 

In  the  above  assumptions  it  was  stated  that  both  v(t)  and  u(t) 
have  identically  zero  means,  using  this  we  now  see  that  the  variances 


estimator  of  the  loss  function  becomes. 
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£,[**»  x(tl)  — "xCtil  t)J  2  =  £|x*  ^x(tx)  -  x(tL  t)  j  [  x(ti)  - 

x(tl  x*  =  ||x*||2r.^  .  . — <  .  m 

£  x ( t L  |  t)  x ( t x  |  t)  T 

where  x(t^  J  t)  is  given  by 

"xUi  I  t)  =  x( t x)  —  ic( t x  \  t) 
and  is  called  the  optimal  error  function, 

Solving  the  optimal  estimation  problem  the  classical  Wiener 
problem  is  formulated. 

Wiener  -  Hopf  Equation 

•  *  A  [  1 

A  necessary  and  sufficient  condition  for  x  »  x(t^  t)  to  be  a 

minimum  variance  estimator  of  x(t^)  for  all  x*(  is  that  the 

matrix  function  ACt^T*)  satisfy  th©  rolafeion 

cov.  x(t1),a(7')J  —  f  A(tlir)cov.[.(r).-(cr)]dr-0..(l. 

to-' 

or  equi  alently 

cov.  x'Ctj^j  t)  ,25(7*)]  -  0  . .....(1.10) 

A  Cotrollqry  from  the  Wiener  -  Hopf  Equation  is 

cov.  ptCtjJ  t)(x'(t1|  t)]  -  0  . .  .(1.11) 

In  Appendix  C  the  Wiener-Hopf  Equations  are  converted  into  a 
system  of  differential  equations,  A  summary  of  these  results  are  as 
follows , 


(1)  The  canonical  form  of  the  optimal  filter. 

The  optimal  estimate  'xCt  \  t)  is  generated  by  a  linear  dynamical 
system  of  the  form 

cfx( t  |  t)  ■=  F(t)'x(t  |t)  +  K(t)z(t  |  t)  . . . .  I 

dt 

where  z(t|t)  *  z(t)  —  H(t)x\t  |  t) 

and  the  initial  state  of  equation  I  is  given  byx-ftol  t(>)  which  in  turn 
must  be  equal  to  zero. 


I 

■  ' 

. 

:  ■ 


MESSAGE  PROCESS  FOP  OPTIMAL 


? 


r* 

e? 

Li- 


84. 


For  Optimal  Extrapolation  (prediction)  we  add  to  equation  I  the  relation 

x(t^|t)  =  cj)(tl9  t)x(t  |  t)  for  all  t^  t  .,.....(1.12) 

The  optimal  filter  which  depicates  the  above  equations  is  shown  in  Fig. 


(III-1.2). 

(2)  The  canonical  form  for  the  dynamical  system  governing  the 
optimal  error. 

The  optimal  error  defined  by  the  Optimal  Estimation  Problem,, 

is  given  by  "x(t  |  t)  x(t)  —  x(t  |  t)  . . (1.13) 

The  canonical  form  is  given  by  equation  XI. 

d*x(t  I  t)  “  F(t)lc(t  |  t)  +  G(t)u(t)  —  K(t)  fv(t)  + 


dt  1 

H(t)  x(tlt)]  . . . . XI 

This  situation  is  depicated  in  Fig.  ( 111-1,3) . 

(3)  Optimal  gain, 

Define  P(t)  -  cov,  px(t  |  t),*x(t  It)  ..............(1.14) 

Then  the  optimal  gain  is  given  by 


K(t) 


P(t)HT  (t)R’1  (t) 
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(4)  Variance  equation 

The  solution  of  P(t)  is  of  central  importance  because  the  form 

of  the  optimal  filter  is  known  (see  Fig. III-l . 2)and  the  only  unknown  is 

the  optimal  gain  which  follows  immediately  once  P(t)  is  known.  The 

solution  for  P(t)  is  given  by  the  nonlinear  Riccati  equation 

dP(t)  -  F(t)P(t)  +  P(t)FT(t)  —  P(t)HT(t)R-1  (t)H(t)P(t) 
dt 

+  G(t)Q(t)GT(t)  ,  IV 
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where  a  solution  of  IV  will  exist  and  be  uniquely  determined  for  all 
t  tQ  by  the  specification  of 

P(tQ)  =  P0  =  cov.  [*x(t0  |  tQ) ,x(tQ  |  t0)  ..........  (1 . 15) 


which  because 

x(tQ  |  tc)  =  x(tQ)  —  x(tQ  |  tQ)  =  x(tQ)  =  xQ 

becomes 


Pq  COV  .  ^  Xq  ,  Xq  J  .oe.09.0.00000.090.0.0.0.  .(1.16) 

because  for  all  |  t  -  tQ  |  sufficiently  small,  given  a  nonnegative 
definite  matrix  PQ,  IV  satisfies  a  Lipschitz  condition. 

The  above  results  are  summarized  in  the  following  theorem. 

Theorem  (Uhl).  Linder  the  assumptions  j  —  A^  j  3  the  solution 
of  the  optimal  estimation  problem  with  tQ  <-«x>  is  given  by  relations 
(I- IV).  The  solution  P(t)  of  IV  is  uniquely  determined  for  all  t >  tQ 
(as  explained  above  by  the  specification  of  P0, 

A  difficulty  in  finding  a  solution  for  P(t)  occurs  when  the 
"steady  state  solution"  of  P(t)  is  treated.  The  steady  state  situation 
solution  occurs  when  t0 oo  ,  physically  this  means  that  an  infinite 
past  of  the  variance  is  known.  If  we  assume  that  steady  state  is  reached 
then  x(t)  is  given  by  equation  (1.5),  but  this  equation  holds  only  if 
the  message  model  is  asymptotically  stable,  hence  Theorem  (HI-1)  no  longer 
applies  and  some  additional  conditions  must  be  imposed  on  the  message 
model . 


In  addition  to  the  fact  that  the  message  model  may  not  be  stable 
is  the  fact  that  H(t)  may  be  singular. 
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The  following  Theorem  gives  two  sufficient  conditions  for  the  steady 
state  problem  to  he  meaningful.  The  first  is  a  natural  consequence 
of  the  definition  for  x(t)  to  be  in  a  steady  state  condition.  The 
second  is  a  weaker  condition  developed  in  Section  1  Part  II. 

Theorem  (JU-2) .  Denote  the  solution  for  P(t)  as 

P(t)  -  TT  (t;P0,c0)  . (1.17) 

then  lim  I  |  ( t ;  0 , t  )  =  P(t) 

t0->  —  o-o 

This  limit  will  exist  if  either  the  following  conditions  hold 

(i)  The  message  model  is  uniformly  asymptotically  stable;  or 

(ii)  the  message  model  is  "completely  observable" 

Proof:  The  proof  of  this  theorem  follows  by  dualizing  -the  result  (3»20) 

for  the  regulator  problem. 

The  matrix  P(t)  is  the  covariance  matrix  of  the  optimal  niter 
described  by  equation  I  when  the  following  conditions  hold. 

(i)  An  arbitrarily  long  record  of  past  measurements  is  available 

(ii)  The  initial  state  x(tQ)  is  known  exactly. 

From  equation  IV  we  see  that  if  the  matries  for  the  message  are 
constants  then  so  will  P(t)  be  a  constant.  Because  the  matries  for  the 
message  process  are  constant  this  in  no  way  implies  that  P(t)  will  be 
a  constant,  but  it  does  imply  that  if  a  limit  exists  then  it  will  be 
constant.  If  the  matries  for  the  message  process  are  constant  then 
in  the  limit  the  problem  reduces  to  the  classical  stationary  Wiener 
problem,  (subject  to  conditions  which  will  be  shown  later). 


89. 


Further  the  variance  equation  suggests  a  simple  method  of  determining 
the  optimal  gain  for  steady  state  cases  with  constant  matries,  this  is 
obtained  by  simply  setting  dP(t)/dt  =»  0  because  P(t)  is  constant. 

The  problem  then  reduces  to  one  of  solving  a  system  of  nonlinear 
algebraic  equations.  A  difficulty  arises  out  of  this  method  however, 
that  is  the  solution  of  algebraic  equations  because  they  are 
quadratic  in  nature  are  not  in  general  unique.  Even  if  P(t)  is 
required  to  be  nonnegative  definite,  the  algebraic  equations  may 
fall  to  have  a  unique  solution,  consider  the  following  example 


F  - 

1 0 

/  1 

G  - 

l1 

°J 

\-l 

H  -  (0  1) 


from  which  P(t) 

can  be  calculated  to 

be,  for 

rll  "  *•  *11  "  1 

2P!2 

+ 

1 

P12  ’  P11  + 

P22 

1  ‘  P12P22 

2P!2 

+ 

1 

a  p2 

22 

Solving  these  equations  one  obtains  the  following  solutions  for  P 


l\/  1  -  1  . 

- 

V  2  +1 

h  +  \j  2  ,  1+  \/i 

P~  - 

-V2  +1, 

\/2  -  1  /  2 

1  +  \/  2  .  1  +  v  2  / 

V  2  +  5, 

\jT  +'  1  \ 

>-1  -  ,  1  + 

-  p4  - 

\/2  +  1.  ) 

V/2  +  1  / 

1  +  V”  ,  -1  v~  / 

and  we  see  that  P]^  is  nonnegative  definite  with  P2  being  positive 
definite,  while  P4  is  nonpositive  definite  and  P3  is  negative  definite. 
Thus  we  see  that  two  nonnegative  definite  solutions  for  P  exist. 
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But  we  see  that  although  the  message  model  is  observable  it  is  not 
controllable.  Thus  as  is  shown  in  Theorem  (3)  the  optimal  filter 
may  not  be  asymptotically  stable.  To  determine  which  of  the  P  ’s 
actually  is  the  limit  of  equation  IV  the  solution  of  IV  can  be 
attempted  and  then  take  its  limit,  this,  however,  is  a  great  deal 
of  work  involving  finding  the  roots  of  a  fourth  order  polynomial 
with  two  complex  roots.  To  check  whether  or  not  the  message  process 
is  stable  we  calculate  its  transition  matrix  to  be 


r  i 

/  cosh(t-tQ)  ,  sinh(t-tQ) 1 

exp. 

[F(t  -  to)] 

ss 

\  slnh(t-tQ)  ,  cosh(t-t0)  j 

from  which  we  see  that  (for  an  initial  xQ  of  1,1  )  that  the  message 
process  is  unstable, 

Before  we  can  proceed  with  the  problem  of  determining  what 
additional  condition  must  be  imposed  on  a  system  for  the  Wiener  problem 
to  be  meaningful  we  dualize  the  stability  theorem  for  the  regulator 
problem, 

Theorem  (JlL3)  :  Assume  that  the  model  of  the  message  process  is 

(i)  uniformly  completely  observable; 

(ii)  uniformly  completely  controllable; 

(Hi )  fii<  II  Q00||</?2.  J3l  £  II  R<0  II  </?4  for  all  t; 
dv)  ||  F(t)|l  <  n5. 

Then  the  following  is  true: 

(i)  The  optimal  filter  is  uniformly  asymptotically  stable; 
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(ii)  Every  solution  ~|  f ( t »  P0,t0)  t^ie  variance  equation(IV) 
starting  at  a  symmetric  nonnegative  matrix  PQ  converges  to  P(t)  as  tQ— 

Thus  we  see  that  if  the  message  process  is  completely  controllable 
and  completely  observable  then  the  filter  is  asymptotically  stable,  and 
a  limit  to  the  variance  equation  exists  as  was  shown  in  Theorem  (Jd . 

The  problem  now  is  when  does  the  set  of  algebraic  equations  obtained  by 
setting  the  right-hand  side  of  XV  equal  to  zero  for  P(t),  give  the  same 
limit  as  for  IV. 

An  immediate  consequence  of  the  above  Theorems  (2)  and  (3) 
is  the  following  result. 

If  we  assume  that  both  Theorem  (2)  and  (3)  hold  and  that  the 
matries  F,  G,  H,  Q,  R,  are  constants 

Then  when  t0=—  go  ,  the  solution  of  the  estimation  problem  is 
obtained  by  setting  the  right-hand  side  of  XV  equal  to  zero  and  solving 
the  resulting  set  of  quadratic  equations,  That  solution  which  is  non¬ 
negative  definite  is  equal  to  P, 

To  see  this,  observe  from  Theorem  (2)  that  the  limit  of  IV 
[?(t)]  exists  and  la  constant.  By  Theorem  (3 ) ,  all  solutions  of  IV 
starting  at  the  nonnegative  matrix  P0  converges  to  P,  Hence  if  P  is 
found  for  which  the  right-hand  side  of  IV  vanishes  and  if  this  matrix 
is  nonnegative  definite,  it  must  be  identical  with  P. 

There  are  cases  when  the  conditions  on  Theorem  (3)  do  not 
hold,  i.e,,  the  filter  may  not  be  uniformly  asymptotically  stable,  but 
we  still  wish  to  solve  the  problem.  This  case  is  shown  in  example  (HT  - 
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and  example  (3).  It  is  required  to  select  which  of  the  solutions  for  P 
gives  the  limit  of  equation  IV.  To  do  this  consider  scalar  function. 


g [x*/x(t  I  t) 


2 


P(t) 


which  in  Theorem  (III  5)  was  shown  to  be  a  Lyapunov  function,,  hence  by 
Lyapunov's  Theorem  (see  Appendix  B)  if  P(t)  is  positive  definite  of 
all  x  =0  then  the  system  is  stable.  Thus  we  can  use  this  criterion 
to  determine  which  of  the  solutions  are  the  actual  limit  P(t)  to 
equation  IV. 

Returning  to  example  £  we  see  that  the  solution  to  equation  IV 
is  given  by  because  it  is  nonnegative  definite*  the  positive  definite 
solution  cannot  exist  because  this  implies  that  the  optimal  filter  is 
uniformly  asymptotically  stable,  which  in  this  case  is  not  so. 

The  only  unknown  quantity  yet  to  find  is  the  cov,  x(t0),x(t0)J 
This  may  be  specified  as  part  of  the  problem.  Or  in  the  case  when  we 
assume  that  the  message  model  has  reached  steady  state  the  cov  [xo,x0] 
is  given  by 

S(t)  ■  cov,  x,xj  * 

$T(t.f)dr 

provided  the  model  is  asymptotically  stable, 

This  expression  can  easily  be  reduced  to  the  differential  equation 

dS  -  F(t) S(t)  +  S(t)FT(t)  +  G(t)Q(t)GT(t)  ......(1.19) 

dt 


oo 


(t,r  )g(T)q(T)°t(T) 

. . ...(MS) 


in  the  case  when  F,  G,  Q  are  constants  this  equation  can  be  evaluated 
by  equating  the  right-hand  side  to  zero. 
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EXAMPLE  PROBLEM  III  -  1 

To  illustrate  the  theory  developed  for  the  stationary  case,, 
consider  the  following  example.  This  example  was  also  solved  by 
Lee(  )using  Spectorical  Factorization. 

2 

The  message  power  density  spectrum  is  given  by  a  » 

b^w2+l 

the  noise  is  white  with  power  density  spectrum  c^. 

Because  the  problem  is  stationary,  we  set  the  right-hand 
side  of  the  variance  equation  equal  to  zero,  resulting  in  a 
quadratic  equation  for  P 

pm  -  2rn  fn  Pn  -  £ti  <iii  e2,  -  o 

let  g  (  ■  Kj,  ®  1  and  solve  for  P,,  to  get 

P“  *  rn  (f,<  *  /f?,  +  £ LU  ) 

'  rn  1 

because  Pjj  is  positive  definite  (or  at  least  nonnegative  definite), 
the  positive  sign  must  hold,  hence 

pu  -  r„  u,  +  /£,v”; . nr  i 

'  r,,  I 

Now  using  the  equation  for  optimal  gain  we  get 
k„  -  f„  +  hi . +  5jli. 

rH 

The  transfer  function  of  the  filter  can  be  found  by  taking 
the  Laplace  transforms  for  Equation  I  (Note  -  the  initial  state  of 
the  optimal  filter  is  zero). 

The  transfer  function  is  given  by 

kn _ 

s  -  f#J  +  k  , , 


■ 

.  '  -  ■  ■  >  ■  -  1. 
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i 

”  i 


-  -  Vf-'V  ■■  ,,r  .?«  , ,  t 

. 

■ 


.  . 


94. 


To  compare  this  with  the  filter  obtained  by  Lee,  we  must 
find  the  relationships  between  a,  b,  c  and  k„  ,  f, ,,  and  q((  . 

Using  the  standard  form  of  input,  output  power  relations 
for  a  linear  system  H(s) 

<3>o0(s)  =  H<-s>  H<s>  $  i-Cs) 


bV+1 


Lt>2+1 


b2 


From  which  we  see  that  q 


li 


% 


u>2+f2 


and  f  i 


=  1  also  r. 


=  c2 


Therefore 


=  I  +  Va2  +  c2 

b  be 


The  transfer  function  for  the  optimal  filter  is 


1 

+  Va2 

+ 

b 

be 

4-  Va2 

+  J 

be 

This 

compares 

with  the  corrected  optimal  filter  obtained  by 

Lee  (16,  Pg401) 

Note:  An  error  in  the  problem  stated  by  Lee  occurs  upon  substitution 
of  A  (bottom  page  399)  into  (29)  page  401)  . 
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EXAMPLE  PROBLEM  III  -  2 


Example  of  Time  Varying  Systems 

For  constant  accelerating  missile 


x(t)  =  cK  x(t)  a  °<L 

and  x(t)  =  t^ 

2 


v?e  wish  to  minimize  the  error  between  £x(t)  -  f ( t ) J  2 

Converting  to  the  message  process  technique,  where  n(t)  is  the  radio 
noise  and  is  an  independent  white  noise  source.  Using  the  notation 
of  Section  III,  cov.  £n(t),n(t)J  =  r  -  <$  (t-T) 

The  inertial  noise  is  just  a  constant,  and  concentrated  at  zero 
frequency  given  by  autocorrelation  function  35  Ni 

Because  x  is  known  exactly,  we  can  forward  it  directly  into  the 


filter , 
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From  the  message  process  j  dx  ■  F  x  +  Gu(t), 

dt 


we  have  the  following  canonical  form 

dx]_  *  x2 

dt 

d*2  "  *  ”  U1  +  *3 

dt 

d*3  *  0 

dt 


Rewriting  the  above  equations  in  Jordon's  canonical  form,  results 
in  the  following  equations. 

*1\  /010\/x1\  /  0 


0  0  1 
0  0  0 


x2 

x3 


«1 


:  .  •  1  :  ■' 


'  \  ■  ::  ;  '  ■  -  .'i.'..'  :  j  5r 

7  •  r;  f,-.  ■  ■: 


?/  ! 


&  *:  0 


The  output  of  the  message  process  "y(t)H  is  related  to 
x(t)  by  the  matrix  H  *  (1,0,0.) 

Solving  the  variance  equation,  given  by  Equation  IV  in 
Section  III,  results  in  the  following  system  of  equations 


To  do 
1.19, 


dpll 

dt 

— 

2P12  -  1 

rll 

dP^ 

dt 

£5 

P22  +  P13  ”  P 

dp13 

dt 

535 

P23  -  P11  P13 
rll 

dp22 

dt 

S3 

2p23  ~  Zni 
rn 

dP23 

dt 

33 

p33  ”  P21  p13 

rU 

dp33 

dt 

SR 

2 

P13 

It  is 

now 

necessary  to  determii 

11 


11  P12 


11 


this,  consider  the  equation  given  in  Section  III  denoted  by 
For  clarity,  we  repeat  equation  1,19  below, 


d_  S(t)  -  FS  +  SFt  +  GQGt 

dt 

Given  that  the  initial  states  x^  «=  X2  “  0,  x3  “  —  results 
in  all  but  S33  being  zero,  Using  this  we  arrive  at  the  following 
equation 

(0  0  0  \  /oio\/ooo\/ooo  \  /o  0 

000  -  00  1000  I-  0  0  0  1  0 

0  0  S  3J  \  0  0  0  J  \  0  0  S33  /  \  0  0  S33/V0  1 

from  which  we  see  that  S33  is  given  by  the  constant  N^. 
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Using  the  above  results  we  can  now  form  the  initial  state 

P(t0)  which  is  given  by 

P(tD)  -  /  0  0  0 

0  0  0 

,0  0  N tl 

Solutions  of  the  nonlinear  Riccati  equation  are  found  by 
considering  the  following  2n  canonical  equations 


1  *l 

o 

o 

o 

h 

o 

o 

1 

1 

rll 

x2 

-1  0  0  0  0  0 

*3 

a 

0  -1  0  0  0  0 

*1 

0  0  0  0  1  0 

0  0  0  0  0  1 

\  *3 

\  0  0  0  0  0  0 

x2 

x3 

W1 

W£ 

w3 


Forming  the  transition  matrix  for  this  set  of  homogenous 
equations,  and  setting  tQ  »  0  results  in  the  following  matrix 


©:t,o) 


-t 

it: 
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0 
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„t2 

t3  \ 

rn 

2rll 

6ru 
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0 

CM 

4J 

1 

-t3 

2ru 

6rll 

24rU 

-t 

1 

t3 

rr 

4^ 

C0 

6ru 

24ru 

120ru 

0 

0 

1 

t 

£ 
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0 

0 

1 

t 
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0 
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from  which  the  solution  for  the  Riccati  equation  is  given  by 

7T(t;P0;o)  =  (-©-21  +  ■£"22po)  (  “^11  +  ‘®12Po)"i 


0  0 
1  0 


Lbt  *--J 

•It4  -t 


+  t _ 

rll 


1  tl2 

t2 

F- 

CM 

-U 

r. 

•U 

-t3 

T~  V 

TZ 

t2 

t3 

t* 

6 

24 

120 

0  \ 

0 

Ni/ 


0 


t2 

2 

t 

1 


0 


-t 


t3Nt 

$rU 

-t4N 


24r 


1 


11 


-l/2t 


2 


1  +  t5Ni 


120ru 

Inverting  the  second  equation,  gives  us  the  solution  to  the 
Riccati  equation,  which  in  turn  determines  the  optimal  gain  of  the 
time  varying  filter, 

»4 


P(t)  - 


/F 

£ 

2 


2 

t2 


|i  \ 


i  / 


The  system  of  differential  equation  which  define  the  optimal 


filter  are  given  below: 
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where  (3  is  given  by 

20  N~ 


For  solution  of  variance  equation  using  an  analog  computer 

). 


see  Fig.  ( 
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EXAMPLE  PROBLEM  III  -3  (Impulse  Response  Solution  see  21,22) 


From  which  we  see  that 
2 

dxa  »  and  dx 

dt  dt 


/  dxl 

o  1  i 

X1 

0 

1  dt 

SB  j 

+ 

UX2 

'  d T 

o 

© 

=**« 

i  X2  1 

^  u^  ' 

The  problem  is  to  determine  the  position  and  velocity  of  a 
moving  particle  when  the  measurements  are  corrupted  by  noise. 

We  assume  that  the  origin  of  the  system  occurs  at  t  »  0 


and  that  all  velocity  is  a  constant 
(  i»e)  G(t)  »  0 

From  IV  dP  ■  FP  +  PFT  -  PHTrlHP 


where  P  -  1 01 1  H  ■  (1,  0) 

loo  I 


2p21 

p22 


1—  pil  pl2 

rll 


:  J  , e>  s  ■  HO’lli i  !• 


0  i 


uMniMKi:  -  tl 

l' 

,1  .  . rio*S+  t 

‘ 

-  '  .  >3  ,  .  c ..  m.  s- aii  * 


The  matrix  coefficient  of  the  Hamiltonian  equation  is 


1/rn 

0 

0 

0 

we  can  arrive  at  the  transition  matrix  (i.e.)  for  t 


©(tto) 


-t 

0 

0 


t 

rU 
.2 


0 

0 


2r 

1 

0 


11 


£ 

2r 


6r 


11 


11 


From 


p(t)  ■  (“©"21  +  "©’22  p0>  +  ©"12  Pq)1 


Therefore 


The  initial  conditions  are  X2<0)  ■  constant 

0  0 

0  P22(0) 

The  solution  for  the  variance  equation  is 


P(t)  - 


+  1 


1  t  ' 

0 

o  ) 

o  1 

0 

P22(0)  ; 

t 

“  1  :*2 

t2 1 
2 

-t^  ; 

( 

0  P22(0) 


11 


!ll 


J 


P22(°)  2  '  t  1 

This  problem  waa  solved  on  the  analog  computer.  The 
results  are  compared  with  the  analytic  solution  in  Fig,  (  ), 
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SECTION  III 

PART  II  DUALITY  RELATIONS  BETWEEN  THE  LINEAR  FILTER  AND 
REGULATOR  PROBLEM 

To  avoid  confusion  in  this  parts  all  equations  and  arguments 
for  the  regulator  problem,  will  be  denoted  with  asterisks. 

Under  these  conditions,  the  equations  for  the  regulator 
problem  now  become 

(i)  Plant 

dx*(t*)  -  F*(t*)  x*  4-  G*(t*)u*(t*) 

dt 

.  *  *  * 
y  =  H  (t  )x 

(ii)  Control  Law 

u*(t*)  -  ~K*(t*)x*(t*) 

(iii)  Riccati  Equation 

-dP*(t*T)  »  P*(t*,T)F*(t*)  +  FT*(t*)P*(t*sT) 

dt 

+  HT*(t*)Qt*)H*(t)  -P*(t*sT)G*(t*)R~l(t*)G^*(t)  P*(t*sT) 
Under  the  duality  relations, 

(a)  -t*  «  t 

(b)  FT*(t*)  -  F(t),HT*(t*)  »  G*(t) ,GT*(t*)  -  H(t) 

the  above  equation  (iii)  becomes 

dP*(t*,T)  -  P*(t*,T)FT(t)  +  F(t)P*(t*,T)  +  G(t)Q(t*)GT(t) 

dt 

-P*(t*,T)HT(t)R“1H(t)P,lf(t,,f,T) 

taking  the  transpose  and  comparing  with  the  variance  equation  of 
the  filter  problem 

dP*T(t*,T)  -  F(t)P*T(t*,T)  +  P*T(t*,T)FT(t)  +  G(t)  Q  GT(t)  _ 
dt 

P*T(t*,T)HT(t)R-lH(t)P*T(t*,T) 
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we  see  that  P*T(t*,T)  =  P(t,tQ) 

Furthermore,  by  substituting  P*(t*T)  into  the  optimal 
control  law,  it  can  be  shown  that  K(t)  (optimal  gain  for  the  linear 
filter  problem  is  equal  to  K*(t,c).  Similarly,  it  can  be  shown 
that  under  duality  relations  the  performance  criterion  of  the  linear 
regulator  problem  is  equivalent  to  the  expected  squared  error  loss 
function  of  the  linear  filter  problem. 

Hence;  we  can  conclude  that  under  the  duality  relations, 
the  linear  regulator  problem  (as  stated  in  Section  II,  Part  3)  is 
equivalent  to  the  linear  filter  and  prediction  problem.  Thus,  all 
theorems  proved  for  the  regulator  problem  must  apply  to  the  filter 
problem. 
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SUMMARY  AND  CONCLUSIONS 

In  this  work,  optimal  control  laws  for  dynamic  systems  were 
synthesized.  The  controller  was  considered  to  be  a  device  which 
manipulated  the  inputs  to  the  system,  being  controlled  in  accordance 
with  a  control  law  specifying  the  values  of  the  manipulated  variables 
as  a  function  of  the  system  state  and  externally  generated  distur¬ 
bances  acting  on  the  system.  The  problem  dealt  with  in  this  thesis 
was  that  of  finding  a  control  law  which  makes  the  system's  behaviour 
in  some  sense  best  subject  to  a  pre-assigned  performance  criterion. 

The  properties  which  a  controller  must  have  in  order  to 
optimize  a  system,  were  derived  in  Section  I,  Part  1  and  2.  It 
was  shown  that  the  control  inputs  u(t)  depended  on  a  set  of 
parameters  A (t)  and  a  known  input  v(t)  together  with  the  current 
state  of  the  system  x(t)  are  the  initial  conditions  of  the  Euler 
Lagrange  equations.  A  (t)  was  to  be  chosen  in  such  a  manner  that 
the  Euler  Lagrange  equations  satisfy  the  predescribed  boundary 
conditions  at  the  future  time  T.  Hence,  it  was  shown  that  the 
optimal  control  law  was  specified  by  a  solution  of  a  two  point 
boundary-value  problem  associated  with  the  Euler  Lagrange  equations. 

In  the  continuous -time  case,  no  general  solutions  for  a 
nonlinear  system  have  as  yet  been  developed,  and  for  that  matter, 
very  few  nonlinear  problems  have  been  solved  in  the  continuous¬ 
time  domain.  One  must  solve  these  problems  by  using  some  type  of 
numerical  techniques.  These  methods  are  beyond  the  scope  of  this 
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report  and  for  this  reason,  no  results  for  solving  nonlinear  systems 
are  given.  General  solutions,  however,  for  linear  systems,  subjected 
to  a  quadratic  loss  function,  have  been  developed  in  Section  II, 

Parts  1,  2  and  3,  Under  the  natural  boundary  conditions,  a  more 
elegant  technique  for  solving  the  linear  two-point  boundary  value 
problem,  was  developed  in  Section  II,  Part  3.  This  method,  initially 
suggested  by  Kalman,  develops  a  differential  equation  to  determine  a 
coefficient  matrix,  denoted  as  the  Riccati  equation  in  Section  II, 
Part  3,  of  the  linear  control  law.  Computationally,  this  method, 
when  applicable,  provides  a  much  easier  solution.  Furthermore,  if 
the  results  are  dualized,  an  analog  computer  can  be  used  to  solve 
the  Riccati  equation  to  determine  the  control  law.  The  accuracy  of 
using  an  analog  computer  for  equations  of  these  types,  is  quite 
satisfactory,  as  illustrated  in  Examples  III  -  1,  2  and  III-  1,  3. 

If  the  solution  for  a  stationary  problem  in  a  steady  state  condition 
is  desired,  one  need  only  equate  the  right-hand  side  of  the  Riccati 
equation  to  zero  and  solve  the  resulting  quadratic  equations  for  the 
nonnegative  definite  solution.  Although  the  system  of  quadratic 
equations  may  be  difficult  to  solve  analytically,  they  are  readily 
solved  using  an  analog  computer. 

In  Section  III,  Part  4,  modern  techniques  are  developed  for 
systems  subject  to  constraints  on  both  the  state  variable  and  the 
control  Inputs.  Some  practical  problems,  using  phase  plane  analysis, 
are  illustrated  for  second  and  third  order  systems  with  nonlinear 
controllers.  These  ideas  are  an  extension  to  the  well-known  bang- 
bang  servo  problem. 
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The  Wiener-Hopf  equations  in  Section  III  are  reduced  to 
a  system  of  differential  equations.  This  has  the  advantage  in  a 
time  varying  system,  of  providing  a  physically  realizable  filter. 

In  the  steady  state  condition  of  a  linear  stationary  problem,  the 
optimal  filter  can  be  solved  by  equating  the  right-hand  side  of  the 
variance  equations  equal  to  zero,  and  solving  the  resulting 
quadratic  equations  for  the  nonnegative  definite  solution, 
Computationally,  this  is  much  easier,  especially  for  complicated 
problems,  than  the  more  conventional  methods  (i.e.,  using  transforms 
and  spectorial  factorization),  Furthermore,  the  methods  presented 
In  Section  III  are  readily  applicable  to  computer  solutions  whereas, 
if  one  used  classical  techniques,  this  would  be  extremely  difficult 
if  applicable  at  all. 

It  is  hoped  that  the  methods  presented  in  this  report  have 
contributed  to  the  synthesis  of  optimal  systems  and  that  they  will 
now  find  practical  application  in  the  design  of  control  systems, 
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APPENDIX  A  NOMENCLATURE 

The  nomenclature  and  symbolism  used  in  this  report  are  consistent 
with  modern  terminology  in  the  field  of  control  systems.  However , 
because  it  is  difficult  in  any  given  report  to  find  a  complete  definition 
of  all  symbols  and  nomenclature  we  enclose  this  appendix. 

Small  letters  containing  the  latter  part  of  the  alphabet,  such  as 
x(t),y(t),  etc.  will  be  used  to  denote  vectors.  Sometimes  an  arrow 
placed  above  or  below  the  letter  is  also  used  However,  because  in  this 
report  no  difficulty  should  be  encountered  determining  whether  or  not 
a  vector  is  used,  the  arrows  are  dropped.  Interminate  small  letters 
such  as  h,  g,  etc.  will  be  used  to  denote  vector  functions,  once  again 
the  arrows  which  are  used  in  some  reports  will  be  dropped.  The 
components  of  vectors  shall  be  denoted  with  subscripts  i.e.,  the 

components  of  the  n  dimensional  vector  x  are  x^,X2» . ,xn,  similarly 

components  of  vector  functions  will  be  given  in  the  same  manner  i.e., 
the  vector  function  f  has  components  ^1,^2 . »fn« 

Scalar  functions  will  be  denoted  by  capital  letters  such  as 
L(x)  ;  V(x,u),  etc,  and  some  small  greek  letters  such  as  efc  . 

Differentiation  of  a  vector  is  denoted  by  one  of  the  following 
methods,  . 


as 


dt 


x(t) 


d_ 

dt 


xi(t) 
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The  derivative  of  a  scalar  function  is  sometimes  denoted  by  the 
symbol  ^7  because  the  derivative  is  just  the  gradient  function,  and  if 
there  is  more  than  one  vector  argument  a  subscript  on  \/^is  used  to 
denote  the  variable  with  respect  to  which  the  derivative  is  taken. 

This  method,  however,  is  awkward  and  the  author  prefers  to  drop  the 
symbol  and  just  use  subscripts  as  shown  below. 


VU*)  =  Lx(x)  “  /  J)_l,  . .  d  L  I 

(t)xl  dx2  bxn  I 


The  differentiation  of  scalar  as  was  pointed  out  is  just  the 
gradient,,  hence  the  differentiation  of  scalar  with  vestor  arguments  is 
vector . 

Matries  are  denoted  by  capital  letters  such  as  A,  B  etc,  and 
most  capital  Greek  letters  i.e.,  ,  (Jj).  The  transpose  of  a  matrix 

is  indicated  as  AT  and  the  inverse  by  A“l.  The  symbol  a^j  denotes  the 
it*1  row  and  the  jth  column  of  the  matrix  A.  The  determinate  of  A  is 
denoted  by  det.  |  A  (  ,  and  adjoint  is  given  by  A. 

The  Jacobian  matrix  (sometimes  represented  by  J(?0)  gives  the 
differentiation  of  a  vector  function.  We  shall  denote  vector  function 
differentiation  as  follows, 


\  / 


fx(x,y) 


(A-l) 
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Norms . 


A  norm  is  a  function  which  assigns  to  every  vector  (or  point) 
x  in  a  given  Euclidean  space  a  real  number  jj  x  ||  such  that  the  following 
axioms  are  obeyed. 

(i)  ||  x  ||  —  0  for  all  x 

(il)  II  x  +  y ||  -  ||  x  ||  + 


II  y  II 


(ill)  ||  CX  x  II  =  |  Of  |  .  ||  x  || 
(Iv)  ||  x  | 


=  0  implies 


for  all  x,y 
for  all  x  and  complex  constant 
x  -  0 


If  (iv)  is  missing  we  say  ||  x  ||  is  a  seminorm. 

Some  examples  of  well-known  norms  are 

n 


(a)  ||  x  || 


s,  i 


CM  l|  x  II  -  £  (  ,}  J 


1/2 


This  is  the  standard  definition  of  the  distance  of  a  point  in 
Euclidean  space  from  the  origin,  also  called  the  Euclidean  norm. 


(c) 


(x^Ax)  1/2  where  A  is  symmetric  and  positive 


definite  ;  called  the  generalized  Euclidean  norm. 


(d) 


Max 

i 


N 


In  explicit  calculations,  ||  x  II  always  means  the  Euclidean  norm. 
This  idea  can  be  extended  to  the  norm  of  a  matrix,  defined  by 
A  ||  =  Min  K  such  that  ||  Ax  ||  U  K  ||  x. 

The  following  inequalities  hold 
(i)  ||  AB  ||  = 
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(ii)  ||  At  ||  =||  A  || 

(iii)  For  the  Euclidean  norm 

II A II 2 


Max  |  xtA^Ax;  x^x  =  1 


A  matrix  A  is  said  to  be  positive  definite  ^positive  semi- 
definitej  if  the  quadratic  form 


xtAx  =  Y. 


xi  aij  xj 


ij  =  1 

is  positive  ^nonnegative j  for  all  x  0. 

Further  it  can  be  shown  that  a  matrix  A  is  positive-definite 
£ semidef inite  j  if  and  only  if  the  following  conditions  hold: 

(i)  There  is  a  nonsingular  singular  matrix  B  such  that 
BTB  =  A 

(ii)  ^  -[(A)  >  0  j^^^(A)  —  oj  for  all  i;  where  ^  -[(A)  denoted 

the  eigenvalue  of  the  n  X  n  matrix  A  and  are  the  roots  of  the  polynomial 
det  (A  -  X  1  )  • 

(iii)  All  principal  minors  of  A  are  positive  nonnegative  . 

Class  of  a  System: 

A  function  f(x)  is  said  to  be  of  class  C1  if 


l 

c>  f 


are  all  continuous. 


c)  Xj  1  <)xj2  . bxji 

A  function  f(x)  is  said  to  be  of  class  if 
i 


bXjl£Xj2 . £xji 


are  all  continuous  except  on  a  set  of  zero  area 


where,  however,  both  left  and  right-hand  limits  exist. 
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Lipschitz  Condition 

If  a  function  f(x.,t)  is  Lipschitz  condition  it  must  obey  the 
inequality 

II  f(x,t)  —  f  (y » t)  ||  -  k  ||  X  —  y  II 

where  k  is  a  positive  constant. 

Expectance  Operator 

The  expectance  operator  is  defined  by  £  ,  from  which  we  define 
the  covariance  of  a  function  to  be  £  4 

cov.  [  x(t)  ,y(  T)  ]  =  £x(t)yT(T)  —  £x(t)  &  T(T) 

where  £x(t)  is  the  mean. 

The  variable  assignment  used  for  the  control  problem  throughout 
this  report  is  as  follows: 

C  Control  law  matrix 

F  Matrix  of  the  state  transition  coefficient  in  the  linear  case 

f  State  transition  function 

G  Matrix  of  coefficients  relating  control  input  to  state  transition 

in  the  linear  case 

g  Function  specifying  terminating  conditions 

H  Matrix  specifying  system  output  in  the  linear  case 

h  Function  specifying  system  outputs  in  terms  of  system  states 

k  Control  law  function 

K  Matrix  of  coefficients  relating  system  disturbances  to  state 


transition  in  the  linear  case 
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L  Integrand  of  the  performance  criterion;  i.e.,  Loss  function 

M  Observability  matrix 

m  Number  of  components  of  the  control  input  vector 

n  Order  of  the  system,  the  dimensionality  of  its  state  vector 

P  Matrix  of  the  Riccati  equation 

p  Number  of  components  of  the  output  vector 

S  Impulse  response  matrix 

s  Running  time 

s  Laplace  transform 

t  Current  time 

T  Time  at  which  the  control  problem  terminates 

u  Control  inputs 

V  Performance  criterion 

V  Lyapunov  function 

W  Controllability  matrix 

W  System  disturbances 

x  State  variable 

y  System  outputs 

z  Desired  output 

Lagrange  multiplier  function 

Transition  matrix  of  the  linear  state  transition 
Specifies  performance  function  at  the  terminal  time 


Hamiltonian 
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The  variable  assignment  used  for  the  filter  problem  throughout 
this  report  is  as  follows: 

F  Matrix  coefficient  of  the  message 

G  Matrix  coefficient  relating  input  to  the  message 

H  Matrix  coefficient  relating  output  to  the  message 

K  Optimal  gain 

M  Observability  matrix 

n  Order  of  the  system,  the  dimensionality  of  the  message 

m  Number  of  components  of  the  input  process 

p  Number  of  components  of  the  output  process 

P  Covariance  matrix  of  the  optimal  error 

P  Steady  state  solution  of  the  Riccati  equation 

S  Covariance  of  the  initial  state  of  the  message 

t  Time 

tQ  Initial  time 

t-L  Prediction  time 

u  Random  input  signal  used  to  generate  the  message 

v  (Jndesired  random  process 

x  Desired  random  process  (message) 

x(tjlt)  Optimal  estimate  of  the  message  x(t^) 

”x(t]Jt)  Optimal  error 
■* 


x 


E 

IT 


Costate  of  the  veetor  function  x 

Output  of  message  process 

Input  to  optimal  filter 

Expectance  operator 

Solution  of  the  variance  equation 
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APPENDIX _ B_ 

METHOD  OF  A  DYNAMIC  SYSTEM 

The  State  Transition  Method 

Microscopic  physical  phenomena  are  commonly  described  in  terms 
of  cause-and-ef f ect  relationships.  In  Newtonian  mechanics,  the  motion 
of  a  system  of  particles  is  fully  determined  for  all  future  time  by  the 
present  position  and  momentum  of  the  particles  and  by  the  present  and 
future  forces  acting  on  the  system.  Future  forces  can  have  no  effect 
on  what  happens  at  present. 

In  modern  terminology,  we  say  that  the  numbers  which  specify  the 
instantaneous  position  and  momentum  of  each  particle  represent  the 
state  of  the  system,  The  state  is  to  be  regarded  always  as  an  abstract 
quantity,  Intuitively  speaking,  the  state  Is  the  minimal  amount  of 
information  about  the  past  history  of  the  system  which  suffices  to 
predict  the  effeet  of  the  past  upon  the  future,  Further,  we  say  that 
the  forces  acting  on  the  particles  are  the  inputs  of  the  system,  Any 
variable  in  the  system  which  can  be  directly  observed  is  an  output . 

The  system,  in  general,  will  be  described  by  the  following 
nonlinear  vector  differential  equation, 

dx(t)  -  f (x(t) ,u(t) ;t)  . . . (B-l) 

dt 

where  x(t)  denotes  an  n  dimensional  vector; 

u(t)  denotes  an  m  dimensional  vector;  and 

f(t)  Is  a  vector  function  of  the  vectors  x(t)  and  u(t) 


as  described  in  Appendix  A, 
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The  vector  differential  equations  can  be  written  as  an  equivalent 

system  of  n  scalar  differential  equations, 

dx  =  f^xp  ...,,x;u  ,  ....,um;t)  . (B-2) 

dt 

where  (i  =  1,2,  . ,n)  and  x^  and  f^  are  components  of  the 

vector  x  and  vector  function  f  respectively. 

Definition:  If  the  vector  function  f(x,u;t)  is  independent  of  the 

forcing  function  u(t)  for  all  t  i.e,,  u(t)  0,  then  the  equation  (B-l) 
becomes 

dx(t)  =  f  (x(t)  ; t)  . . . . . (B-3) 

dt 

and  the  system  is  said  to  be  free  (unforced). 

Definition:  If  the  vector  function  in  addition  to  being  free  is 

also  independent  of  t  then  equation  (B-l)  becomes 

dx(  t)  =  f  (x(  t) )  . (B-4) 

dt 

then  we  say  that  the  system  is  autonomous . 

The  output  of  a  system,  denoted  by  y(t),  is  determined  by  a  set 
of  algebraic  equations  defined  by  the  vector  equation 

y(t)  »  h(x(t)  ; t)  . . . (B-5) 

where  y(t)  is  an  p  dimensional  vector;  and 

h  is  a  vector  function  of  the  vector  x(t) . 

Linear  differential  equations  are  expressed  in  vector  form  as 
a  system  of  linear  first  order  equations  as  follows, 


dx(t)  -  F(t)x(t)  +  G(t)u(t)  . (B-6) 

dt 

and 

y(t)  -  H(t)x(t)  . . (B-7) 


'  ■  ' 
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where  F(t)  is  an  n  x  n  dimensional  matrix; 

G(t)  is  an  n  X  n  dimensional  matrix;  and 

H(t)  is  an  n  X  p  dimensional  matrix. 

The  above  vector  equations  are  equivalent  to  a  system  of  scalar 

equations  written  as  follows: 

*2,  m 

=  L  fijXj  +  £  giju  . (B-8) 

dt1  j=l  J  J  j-1  J  J 


P 

£ 

j=i 


hijxj 


(B-9) 


where  i  = 

1,2,... 

If  we  assume  that  f(0,0,t)  = 

0  then 

Jacobian  matries 

bft(t) ' 
b  xj(fc) 

and 

’  dMt)’ 

obtained  by  the 


standard  linearization  method. 

The  solution  of  the  equation  (B-6)  is  denoted  by  ^  (t;xQ,t0), 
where  xQ  is  the  initial  state  at  time  tQ .  Then  the  solution  can  be 
expressed  by  the  well-known  form. 

9(t;x0,t0)  =  (5  (t,t0)x0  +  (t<<)6(tf)u<tf)d(r...(B-10) 

where  (^)  (t,tQ)  is  the  so-called  transition  matrix  which 
obeys  the  following  conditions. 


(i)  d$(t.CT )  -  F(t)  (f>  (t,cT) 

dt 

subject  to  the  condition  that  ^  (t,t)  =  I,  where  I  denotes 


the  unit  matrix. 
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(ii)  (])  (t2,t0)  =  (£)  (t2,ti>  <^)  (ti,tQ)  for  all  tQ>t1,t2 

( iii)  $  _1(t,t0)  =  (^)  (tQSt)  for  all  t,tQ. 

(iv)  In  the  case  where  F  is  a  constant  matrix  then  the  solution 
for  the  transition  matrix  is  given  by, 

(t,tQ)  -  exp.  ft  -  tQ]  F  =  Ys  (  [t  -tj  f)  k 

^  L  J  k=0  k.'° 

The  Impulse-Response  Matrix 

In  the  filter  and  predication  problem  a  system  of  linear 
differential  equations  is  developed  from  the  well-known  Wiener  equations. 
In  effect  this  is  the  problem  of  realizing  a  linear  system  of  differen¬ 
tial  equations  from  the  systems  impulse  response,  for  this  reason  we 
include  some  results  of  identification  theory.  The  theory  of 
identifying  a  system  of  differential  equations  from  an  impulse  response 
is  a  broad  and  difficult  field  in  itself,  for  this  reason  a  complete 
study  is  not  attempted  here. 

We  now  define  a  dynamic  system  from  the  point  of  view  of  impulse 
responses.  Consider  a  system  which  is  at  rest  at  time  tG;  i.e,,  one 
whose  inputs  and  outputs  have  been  identically  zero  for  all  t^tQ.  We 
apply  at  each  input  in  turn  a  very  sharp  and  narrow  pulse.  Ideally, 
we  would  take  uj_J(t)  =  (3  ijc)^  ”  to)  »  where  ^  *-s  the  Dirac  delta 
function,  ^  ij  is  the  Kronecker  symbol,  and  1  If  i,  j  <  m.  We  then 
observe  the  effect  of  each  vector  input  u-^(t)  on  the  outputs.  The 
matrix  S(t,tQ)  =  s  ^  j  ( t ,  t0)  J  so  obtained  is  called  the  impulse 
response  matrix.  Since  the  system  was  initially  at  rest  we  must  define 
S(t,tQ)  ==  0,  for  all  t  <  tQ. 


:  ...  "'I 
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With  these  conventions,  the  output  of  a  linear  system  originally 
at  rest  is  related  to  its  input  by  the  well-known  convolution  integral: 

y(t)  -  f  s(t,r)o(T)dT  . B-ii) 

coJ 

Unfortunately,  this  definition  does  not  explain  how  to  treat 
systems  which  are  not  initially  at  rest. 

If  we  suppose  that  system  governed  by  equations  (B-6,7)  are 
initially  at  rest  i.e.,  xQ E  0  ,  then  the  impulse  response  and  the  state 
transition  are  related  by  the  following  equation. 

S(t ,T)  -  H(t)^)  (t,r)G(T)  t  =  r  . (B-12) 

=  o  t-=r 

From  which  we  see  that  it  is  just  a  matter  of  substitution  to  find 
the  impulse  response  once  the  terms  on  the  right-hand  side  are  known. 

If,  however,  we  attempt  to  determine  the  system  of  differential  equations 
from  the  impulse  (experimentally  observed)  the  problem  is  no  longer 
trivial.  Even  if  we  succeed  in  determining  a  system  which  obeys  equation 
(On)  one  must  still  ask  the  question,  is  the  system  of  the  lowest 
possible  order.  This  can  be  stated  in  other  words  as,  have  we  realized 
an  irreducible  identification  of  S(t,T)*  We  may  now  ask  ourselves  the 
question  ’’When  and  how  does  the  impulse  matrix  determine  equations  of 
the  system?" 

The  first  condition  we  consider  is  stated  as  follows: 

An  impulse-response  matrix  S(t,T )  is  realizable  by  a  finite  - 
dimensional  dynamical  system  (B-6,7)  if  and  only  if  there  exists  continuous 
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matrices  P(t)  and  Q(t)  such  that 

S(t ,f)  =  P(t)Q(f)  for  all  t,  . . . (B-13) 

This  result  was  assumed  to  be  true  by  Shinbrot  ^2oj  in  his  development 
of  an  optimal  time  varying  filter. 

In  many  cases  the  impulse  response  is  stationary  and  can  be 
given  by  S(t -T ),  which  in  turn  can  be  represented  by  its  Laplace 
transform  Z(s)  =  J^S(t). 

Another  very  important  condition  that  has  been  proved,  at  least 
the  stationary  case,  J^l 2  23 J  is  stated  below. 

Knowledge  of  the  impulse-response  matrix  S(t,7")  identifies  the 
completely  controllable  and  completely  observable  part,  and  this  part 
alone,  of  the  dynamical  system  which  generated  it,  This  part  has  the 
smallest  dimension  among  all  realizations  of  S(t,7^)  and  is  identified 
by  S(t,7"  )  uniquely. 

Stubberud  23  has  using  different  techniques  shown  that  a 
reducible  system  can  be  transformed  into  the  same  form  as  that  stated 
in  Section  I,  Part  2  to  define  an  uncontrollable  system,  hence  we  can 
conclude  on  the  basic  of  Stubberud 's  results  that  because  equivalent 
forms  are  used,  that  controllability  and  observability  must  also  be 
conditions  which  must  be  obeyed  if  an  irreducible  realization  is  to 
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Stability  of  a  Dynamic  System 

Optimal  systems  defined  by  their  optimal  control  law  are  no 
guarantee  that  the  system  is  stable.  Stability  must  be  checked  by 
using  one  of  the  following  methods  depending  on  the  complexity  of  the 
problem.  If  the  problem  is  linear  with  constant  coefficients  Routh's 
Stability  Criterion  may  be  used.  This  theorem  is  well  known  in  linear 


control  systems  and  is  stated  in 


If  the  system  is  linear 


with  non-constant  coefficients  the  following  theorem  may  be  used. 


For  a  formal  proof  see 


[  ] 


Theorem  (B-l),  Consider  a 


continuous  time,  linear  dynamic  system  (B-6) . 

dx(s)  =  F(s)x(s)  +  G(s)u(s) 
ds 

subject  to  the  restrictions 

f  c  <  cxj  for  all  s 


c3  C  oo 


(i) 

||  F(s) 

(ii) 

0  <  c2 

for  all  x 


=  I,  all  s 


Then  the  following  propositions  concerning  this  system  are 
equivalent 

(A)  Any  uniformly  bounded  excitation 
II  u(s)  ||  =  c4  <  ^  (  s  =  to 

gives  rise  to  a  uniformly  bounded  response  for  all  s  ^  tQ 

s 

II  *00  II  =  ||  $  (s,t0)x0  +  Jc£>  (s,£f)G((j  )u((f  )d(j\\ 

=  c5<c4*  II  xoH>  < 


(B- 14) 


O-o 


(B)  For  all  s 

s 


=  to* 


f  ||$  (*»(?)  II  d(f  =  c6 


<  OO 


(B-15) 
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(C)  The  equilibrium  state  "  xe  "  equal  zero  of  the  free 
system  is  uniformly  asymptotically  stable; 

(D)  There  are  positive  constants  Cy,Cg  such  that,  whenever 

s±t0»  ||  $  (s,tQ)  ||  _<  c7exp  £-c8(s  -  tQ)]  . . . (B-16) 

(E)  Given  any  positive  -  definite  matrix  Q(s)  continuous 
in  s  and  satisfying  for  all  s  >  t^ 


0  <  c9  <_  Q(s)  <  c  < 

io 

the  scalar  function  defined  by 


(B-17) 


V(x, s) 


r  00 

J  l|<T«7'.Ox||2  dCf 

Q<05 


F(s) 


(B-18) 


exists  and  is  a  Lyapunov  function  for  the  free  system  satisfying  the 
requirements  of  Lyapunov's  Theorem  (see  ),  with  its  derivative 
along  the  free  motion  starting  at  x,s  being 


V(x, s) 


*11 


Q(s) 


None  of  the  propositions  in  Theorem  (  )  offers  a  method  of 

determining  whether  a  given  system  is  uniformly  asymptotically  stable 
or  not  without  computing  its  transition  matrix  for  all  s,tQ.  For 
this  reason  it  is  generally  difficult  to  determine  the  stable  of  a 
time  varying  system  without  doing  a  great  deal  of  work. 

In  the  case  of  nonlinear  equations  the  question  of  determining 
stability  of  a  system  becomes  extremely  difficult,  as  of  course  are 
the  solutions  of  a  nonlinear  system.  In  this  paper  a  direct  method  of 
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determining  stability  suggested  by  Lyapunov,  called  the  second  method 
of  Lyapunov,  will  be  used.  Unlike  Lyapunov's  first  method  which  only 
give  information  on  asymptotic  system  about  a  point  the  second  method 
can  be  used  to  determine  various  types  of  stability.  The  difficulty 
of  the  second  method  will  become  apparent  upon  statement  of  the  theorem. 

Lyapunov's  Second  Method 

The  second  method  is  based  on  the  idea  that  if  one  is  able  to 
find  a  suitable  continuous  scalar  function  V(x,s)  which  is  positive 
for  x^O  and  zero  for  x  =  0,  then  along  a  trajectary  x(s)  an 
inequality  of  the  form  V(x(s))_<_  will  guarantee  a  related 
inequality  x(s)  <_  €  with  £  — *- 0  as^— *-0.  The  quantity  V(x)  is 
then  a  measure  of  the  smallness  of  x. 

Theorem  (B-2) 

Consider  the  continuous -time ,  free  dynamic  system 

dx  -  f(x,s)  . (B-19) 

ds 

where  f(0,s)  =*  0  for  all  s. 

Suppose  there  exists  a  scalar  function  V(x,s)  with  continuous 
first  partial  derivatives  with  respect  to  x  and  s  such  that  V(0,s)  =  0 
and 

(i)  V(x,s)  is  positive  definite;  i.e.,  there  exists  a 
continuous,  nondecreasing  scalar  function  such  that  (X(0)  =  0  and, 
for  all  s  and  all  x^O 

0<°^(||x||  )  <  V(x,s)  (see  Fig.  B-l) 
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(ii)  There  exists  a  continuous  scalar  function  /  such  that 
7(0)  =  0  and  the  derivative  V  of  V  along  the  motion  starting  at  s,x 
satisfies,  for  all  s  and  all  x  0, 

V(x,s)  »  dV(xt  s) 

»  lim  £  V(x  +  hf(x,s),s  +  h)  —  V(x,s)  j  /h 


d  V  +  (gradV)^f (xs s) 

Ts 


(B-20) 


-  -7  (l!xli)  <  0 

(iii)  There  exists  a  continuous,  nondecreasing  scalar  function 

such  that  J3  (0)  =  0  and,  for  all  s, 

V(x#s)  £  J]  j||x||] 

(iv)  OC  (  ||  x  ||  ]  — oO  with  ||  X  1 1  — *-  Of 

Then  the  equilibrium  state  xe  ■  0  is  uniformly  asymptotically 

stable  in  the  largej  V(x,s)  is  called  a  Lyapunov  function  of  the 
system  (B-21) , 

The  following  conditions  are  sufficient  for  the  various 
weaker  types  of  stability: 

(a)  Uniform  asymptotic  stability:  (i-ii-111) 

(b)  Equlasymptotic  stability  in  the  large:  (i-ii),  (iv). 

(c)  Equlasymptotic  stability:  (i-il), 

(d)  Uniform  stability:  (i),(iii),  and  V(x,a)  0  for  all  x,s 

(e)  Stability:  (i)  and  V(x,a)  £  0  for  all  x,s. 
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For  proofs  sec  ill) 


DEFINITIONS 

«x(NI) 


OF  V(*,  s), 

,  P(IWI) 


APPENDIX  C 


Results  from  Wiener  Equation 

The  Wiener  Equation  is  given  by 


cov 


x(t),z ((f)  -  J  A(tjJ')  j^cov  [zCp,z((f)]  &T . (c-1) 


and  is  a  necessary  condition  for  an  optimal  system. 


The  optimal  estimate  x(t 


t)  is  given  by 


t  r 

(t  ,  |  t)  -  J  A(t(  ,(T)x(C^)d (/ . (c-2) 


t  ^  O'  <  t0 


where 

with  the  property  that  the  expected  squared  error  in  estimating  any 


linear  function  of  the  message  is  minimised 


[x?x(t( ) 


A  . 

x(t,  |  t) 


1  9  if 

=  minimum  for  all  x 


To  develop  the  associated  differential  equations  as  given  in 
section  (3);  proceed  as  follows. 


let  "t  j  =  t"  and  differentiate  (c-1)  with  respect  to  "t" 


L.H.S: 


d 


d 


cov  £x(t)  ,z(cf)]  -  C°v[-^-  x(t)  ,z(Cf)J 


(C-3) 


and  upon  substitution  of  equation  (  B-6) into  equation  (c-3)  we  get 
L.H.S:  F(t)cov  [x(t)  , z((/)j  +  G(t)  cov  u(t)  ,z  (O')  ]  . (C-4) 


R.H.S: 


C> 


f  A(t,T)  cov  [z(T')  ,z(Cf)  1  d  X 

V 


cov 


where  from  z(t)  -  y(t)  +  v(t)  we  see  that  the 
Jy(*r)  +  V(T)  J  [y  (0")  +  v((^)J  -  cov|y(f  )  ,y(CT)j  +  cov[ 


v(T),v(Cf)] 


(c-5) 


. 

. 


. 
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because  the  system  is  uncorrelated  i.e.  cov 
6 


R.H.S: 


[y(T> ,v(cf)]  -  o 

I —  f  A(t  >T)  cov  fy(T),y(Cf)  1  dT  +  ^ACt,^  )R(C?)  . ...(c-6) 

t  tcJ  L  J  dt 

c  A(t,r)cov  [*(T)m6)}  d T+  A(t,t)cov£ 

y«0,y(CT>]  . (c-7) 

We  see  that  the  last  term  of  equation  (c-4)is  zero  because  of  the 
independence  of  u(t),v(C^)  and  x((j)  when  (j  <  t 
Further  we  note  that  by  replacing  y(t)  byjz(t)  —  v(t)j the  cov £ 
y(t),y(Cf)]  becomes 

cov  [ (y (t) ,y((j )]  “  cov  j^z(t)  ,z  (O')]  —  cov  [ v(t) ,v((7)]  . . . . (c-8) 

and  using  the  fact  that  y(t)  =  H(t)  x(t)  we  further  rearrange  covj^y(t), 
y(CT )  ]  to  be  cov  [^y(t)  ,y(Cf)  j  *  cov  [H(t)x(t),  z((j)  —  v(C^)] 

=  H(t)cov  [x(t)  ,z(CT)  j  —  cov|(y(t)  ,v(^T)  J  ...(c-9) 
We  summarize  important  results  at  this  point 
From  (c-4)we  see  that 

-  cov  x(t)  ,z(^)j  *  F(t)  cov  [  x(t)  ,z(CT)  j 

From  (c-l)the  Wiener  equation  is 

^x(t),z(0")j  -  f  A(t,r)cov[z(r).z(cr)]  dT 


j} 


cov 


From  (  c-7)we  see  that— ~-cov  £  x(t),z(^)j  is  also 


A<t»  T)cov  [  )  ,z((j  )j  d T  +  A(t,t)cov  J  y(t)  ,y((j) j 

and  from  (c-9)  we  get 

cov  Jy(t)  ,y((>)j  -  H(t)  cov  £  x(  t)  ,  z  ((j  )  j  where  cov  £ y (t)  ,  v((^)J 
for  t  >  (j 
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( c- 10) 


Now  making  use  of  the  optimal  estimate  x(t  |  t)  =»  f  IjA(ti'J') 

|  V 

z(J')ld7  consider  (I)  equate  (c-4)and  (c-7)to  get  (c-10, 

F(t)cov  ^  x(t)  ,z((j  )j  -  A(tsT)cov  £  z(T')  ,z((j)  j  dT 

A(tst)H(t)cov £  x,z  (O')]  . . . 

rearrange (c-lO)and  replace  cov  £x(t),z  ((>)]  by  JU,r, 

COV  [*(?■). *((?■)]  dT  from  Wiener's  Equation  to  get 

F(t)  —  A(t,t)  H(C)  j  j  A(t,T)cov  z(T  )z(  )  dT 

d  =  J-^~-A(t,r  )cov[  zCD.zCO')]  dT  . (c-U) 

Taking  everything  under  the  integral  and  factoring  out  cov  j^zCj*), 
*<0")]  equation (c-11)  becomes 

f\  F(t)A(t,7")  —  A(t,t)H(t)A(t,f  ) - ^.A(t,f)  covf 

t  J  !  7)  t  J  L 


*cr).*<o')]  dT  =  0  because  cov  £z(T  ) , z(£f) j  is  non  zero  we  have 

that[F(t)A(t,f)  -l-ACt.j')  __  A(t,t)H(t)A(tT)l  -  0  . (c-12) 

L  O  J 

for  all  values  of  1 0  <_  T  t 

If  R(t)  is  positive  definite  in  this  interval  then  equation  (c-13)  is  a 
necessary  condition. 

We  now  differentiate  the  optimal  estimate  to  get 


f  A 

d_  x(i 
dtL 


t) 


r  >*(T>dr 


r^Act, 

tj  t 


T)z(T)  dT  +  A(t,t)z(t) 


(c-J5) 
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We  now  define  K(t) 


A(t,t)  . . . .  (c-16) 


and  substitute  the  value  for-— —  A(t,T)  from  equation (c-13)into  (c-15) 

u  t 

resulting  in 


A 

d  x(t  t) 
dt 


Jl 


F(t)A(t  ,r>  —  K(t)H(t)A(t,r  )  z(T) 


d  T  .  ...(c-17) 


+  K(t)z(t) 

factor  out  A(t ,T)  and  notice  that  /  A(t,T)z(T)  dT  occurs,  so 

tcr 

replace  this  with  x(t  |  t) ,  thus  equation  ( c-17)becomes 


A  i 

d  x(t  t) 
dt 


F(t)x(t  t)  +  K(t) 


z(t) 


H(t)x(t  t)j  . (c-18) 


this  equation  is  denoted  I  in  section  (III) 

We  now  let^z(t|t)  *  z(t)  —  H(t)x(t|  t)  . (c-19) 

so  I  becomes 

A  ,  A 

d_  x(t|  t)  «  F(t)x(t|t)  +  K(t)z(t|t) 
dt 

^  A 


To  find  the  optimal  error  x(t| t)  *  x(t)  —  x(t|t)  . (c-20) 

we  differentiate  x(t  t)  with  respect  to  "t"  to  get 

d_^c(t|t)  »  d  x(t)  —  d  x(t|  t)  . (c-21) 

dt  dt  dt 

and  from  I  and  equation  (B-6)  we  get 

d_x(t| t)  -  F(t)x(t| t)  +  K(t)  [  z(t)  —  H(t)x(t 1 1) 1  —  [  F(t)x(t)  — 
dt  L  J  L 

G(t)u(t)|  . (c-22) 


“  F(t)x(t| t)  +  G(t)u(t)  —  K(t)H(t)x (t  t)  —  K(t)v(t) 
which  is  denoted  in  section  (III)as  II 
collecting  terms  we  get 

dx(t  t)  -  [ F(t)  — K(t)H(t) 1  x(t| t)  — K(t)v(t)  +  G(t)u(t) 

dt  L  J 
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We  now  obtain  an  expression  for  K(t).  To  do  this  we  observe 
from  (c-23)  that 


cov 


[x(t),y((j)j  -  J  A(t,  T)cov  £  z(  T)  ,z((j  )]  d  T 

and  we  also  note  that  cov  [^(t|t),z(Cf)]  «  0  . (c-24) 

Equation(c-24)becomes  from  z ((j)  *  y ((j)  +  v((j) 

■  COV  J'xCtl  t)  ,y(0/)j  +  cov  [x(t|  t)  tv((j  )  j  . (c-25) 

The  second  term  is  zero  because  of  independence  of  StC 1 1 1 )  and  v(CT) 
We  also  know  that  cov  £x(t),z ((j)  ]  -  cov  ^x(t)  ,y(CT )]  + 


cov  ^x(t),v(^)j  . (c-26) 

the  second  term  of  which  is  also  zero  because  of  independence  of 
x(t)  and  v((j ) . 

Therefore  the  Wiener  equation  becomes 


cov  [x(t),y((7)]  —  f  A(t,r>cov[y(r),y(C?)]  dT» 

. (c-27) 

Since  both  sides  are  continuous  functions  of  "(j"  the  equation  must 
also  hold  when  "(jm  t"  letting  "(jm t"  and  equating  equation  (c-27) 
and  equation  (c-25)  we  get 

cov  pT(t|  t)  ,y(t)  j  «  A(t ,  t)R(t)  . (c-29) 

because  y(t)  *  H(t)x(t)  =  x(t)H(t)^  Equation  (c-29)becomes 

K(t)R(t)  “  cov  j^x(t|  t)  ,x(t)  jll(t)T  . . . (c-30) 

At  this  point  we  state  and  prove  a  corollary  to  Wiener's 
equation  which  will  be  used  in  the  development  of  equations  stated 
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in  section  (III) 

Corollary  (  ) 


The  cov  ^^x(t|  t)  jX^t  j  t)  j  =  0 


Proof.* 

From 


we  see  that 


cov  pc(t|  t)  ,z(^)  j  *  0  for  all  t0  —  ^  ~  1 

and  from  x(t|  t)  =  f  A(t,T  )z(T)  df 

V 

x(t|t)^(t|t)l  =  T^(t|t>A(t,r)Z(r)  df 

V 

and  if  A(t,  )  is  symmetric  we  can  write 
PxCtlt)  )£(t|t)]  -  f £x(t|  t)^z(r)A(t,r  )]  d  T 

<v 

Taking  the  cov.  of  both  sides  we  see  that 


cov 


[  x(t|  t)  /x(t|  t)j  =  0  Q.E.D. 

From  the  above  corollary  we  further  see  that 

[  x(t|  t)  ,x(t)j  =  cov  Jx(t|  t)  /x(t|  t)j  . . 

Upon  substitution  of  equation(c-31)  into  equation(c-30) we  get 
K(t)R(t)  “  cov  x(t|  t)  ,x(t |  t)j  H(t)T  . 


cov 


Now  define  P(t)  =  cov  1 1  t)  /x(t|  t)j 

therefore  equation  becomes 
P(t)H(t)T  =  K(t)R(t) 


(c-31) 

(c-32) 

(c-33) 

(c-34) 


and  if  R(t)  is  positive  definite 

K(t)  =  P(t)H(t)^R(t)  which  is  denoted  in  section  (III) by  III 
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We  now  develop  the  Variance  Equation 


Summary 

I  d x ( t 1 t )  =  F(t)x(t|t)  +  K(t)z(t|t) 

dt 

II  dx(t I t)  =  f F ( t )  —  K(t)H(t)l  x(t  t)  —  K(t)v(t) 

dt  J 

+  G(t)u(t) 

III  K(t)  =  P(t)H(t)TR(ty ' 

We  let  m  (t,']')  be  the  common  transition  matrix  of  I  and  II 

So  the  solution  of  this  system  will  become 

x(t|t)  =  Vj/(t,t0)7(to  |  to)  +  U  ( t , J'  )  f  G(T)u(7')  — 

1 0J 

K(f)v(T)]  df  . . . (c-35) 

We  wish  to  establish  an  equation  to  determine  P(t)  therefore  we  must 


square  equation(3-35)  and  take  the  cov,  of  both  sides 

cov^(t|t)  —  \j/(t,t0)x(t0  I  tQ)j  •['x(t|t)  —  y(t,t0)1T(t0  I  tQ)J 

.  g 


Jf(t,r)[G(r)»(r)  —  kctmt)]  ir  T vj/ct,cT) 

[g(^T)u((5')  —  K(C^)v(CT)  j  d(j 


(c-36) 


J  'f(t.r)  [g(T)u(T)  —  K(T)v(f)]  dT  J[u(0')TG(0')T 

—  v(CT)tK(CT)]  Y  (t,CT)T  d  O'  . (c-37) 

because  u(t)  and  v(t)  are  uncorrelated  white  noise  the  integral  reduces 
to 

tr  f 

G(r>Q(r)G(r> 


P(t)  —  V|/(t,t0)p(t0)  Vjy (t, t0)T  -  Jfy(t,r>[ 
+  K(f  )R(7')K(7')T]  4/(t,f)T  d  x  . 


(c-38) 


-  [  :  ■  '  : 

:•/  -  ■  1  r:  •  .  ,-jf-  ■£$  •/  j'W  ■■Vi 

’  ■  :  ■  ■  : '  :  >. 

[o1'  .  '>^V.  )X.  —  •  ! :y  ;  *  ,  i  •>”  ,>v '  .  .  3 


136. 


let  the  bracket  be  C(t)  and  differentiate  with  respect  to  "t" 

Jl  *  <t,f)  c<r )  4/(t,r)T  <jt  +  J\j/<t,r>c(r> 

■^-'t'(t,7')T  df  +  U/(t,t)C(c)  ^(C,C)T  . . (c-39) 

o  t 

where  from 

d^(t.T)  “  [f(C)  _  K(t)H(t)1  \v  (t,f) . (c-40) 

dt  J 

and  because  V  (t,t>  =  i 

f  [  r(t)  —  K(t)H(t>]  ^  (t,r)ccr)  \)/(t,r)T  dT  +  c(t>  + 

V 

f  ^  (t,r)C(T)  ^  (t,r>T  df[F(t)T  _  H(t)TK(t)T]  . (C-41) 

to*' 

From  equation  (c-38)equation  (c™41)becomes 
-  [f(0  —  K(t)H(t)]  [  P(t)  _  \£/(t,t0)P(t0)VJ/  (t,t0)T]  +  C(t) 

+  [P(t)  —  4/(C,t0)P(t0)y  (t,t0)T  ][  F(t)T  _H(t)TK(t)]  ...(C-42) 

We  now  differentiate  the  L.H.S. 

dP(t)  _  iL  W(t,t0)  P(t.)  U/(t.t0)T  —  U/(t,to)P(t0) 

dt  be  A  A 

y(ttt0)T  -  dP(t)  —  [f(0  —  K(t)H(t)]  [  Vj/  (t,t0)P(t0) 

VJ/(t,t0)Tj  . (c-43) 

—  Vj/(t,to)P(t0)  V[/(t,t0)T  [F(t)T  _  H(t)TK(t)T] 

Equate  equations  ( c-43)and  equations (c-42)and  cancel  terms  results  in 

dP(t)  -  F(t)P(t)  +  P(t)F(t)T  —  K(t)H(t)  —  P(t) H(t)TK(t)T  + 
dt 

K(t)R( t) K(t)T  +  G(t)Q(t)G(t)T  . (c-44) 
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This  equation  is  denoted  as  IV  in  section  (III) 

We  now  consider  the  problem  of  prediction  and  find  a  form  for 

the  optimal  estimate  problem  when  "t,  >  t  " 

The  solution  of  the  equation  dx(t)  =  F(t)x(t)  +  G(t)u(t) 

dt 

is  given  by 

0(t,;t,x(t)>  =  C£>(t,,t)x(t)  +  j  <|>  (t,,  £)G(f)u<*  >d/ 

Since  u(j*  )  for  t  <T  %-  <  tj  is  independent  of  x(t)  in  the  interval 

t0  <  t  <  t  it  follows  from  Wiener's  equation  that  the  optimal  estimator 
for  the  right-hand  side  above  is  zero.  Hence 

x(t,  |  t)  =  (t,  ,t)x(t(|  t)  ( t , >  t)  . (c-46) 

This  however  is  not  true  for  t(  <  t  because  of  the  lack  of 
independence  between  x( ^ )  and  u(J°). 
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